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PREFACE 


This  paper  was  prepared  for  the  Deputy  Under  Secretary  of  Defense  (Environ¬ 
mental  Security),  under  a  task  entitled  “Assessment  of  Traditional  and  Emerging 
Approaches  to  the  Detection  and  Identification  of  Surface  and  Buried  Unexploded 
Ordnance.”  Earlier  work  under  a  task  for  the  Director,  Test  Systems  Engineering  and 
Evaluation,  has  been  incorporated  as  well. 

We  greatly  appreciate  the  comments  of  Dr.  Regina  Dugan,  Ms.  Christine  Jordan, 
and  Mr.  Joseph  Stahl.  Their  efforts  greatly  improved  the  quality  of  the  paper. 
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I.  INTRODUCTION 


Unexploded  ordnance  (UXO)i  contamination  is  a  worldwide  phenomenon.  Much 
of  the  contamination  is  a  legacy  of  wars  in  Europe,  Asia,  and  the  Middle  East.  In  these 
regions,  unexploded  ordnance  includes  bombs,  rockets,  artillery  shells,  landmines,  naval 
ordnance,  incendiary  munitions,  and  pyrotechiuc  devices  used  by  all  combatants.  Wifliin 
the  continental  United  States  (CONUS),  however,  UXO  contamination  problems  are  the 
result  of  research,  development,  testing,  manufacturing,  stockpiling,  and  training  activities. 
UXO  consists  primarily  of  ordnance  items  of  domestic  manufacture.  Even  within 
CONUS,  the  problem  is  large:  current  estimates  indicate  more  than  11,000,000  acres  of 
land  may  be  contaminated  with  UXO  (see  Table  1). 


Table  1.  Amount  of  Potentially  UXO-Contaminated  Land 


Number  of  Acres  Potentially 

Government  Lands 

Contaminated  with  UXO* 

Active  DoD 

6,098,019 

Department  of  Interior 

5,111,521 

Base  Realignment  and  Closure  (BRAC)  Sites 

84,106 

Total 

11,293,546 

*  Does  not  include  Air  Force.^ 


Before  the  mid  1980’s,  UXO  contamination  within  CONUS  was  not  a  major  issue 
of  high-level  concern.  Most  of  the  contamination  was  confined  to  DoD-controlled  military 
ranges  or  stockpile  areas  on  DoD  bases.  Only  military  personnel  or  contractors  were  likely 
to  come  into  contact  with  UXO.  It  was  assumed  that  all  such  persons  would  have  some 
level  of  training  sufficient  to  help  them  avoid  obvious  UXO  hazards  and  seek  help  upon 
their  discovery. 


^  For  purposes  of  this  paper,  unexploded  ordnance  (UXO)  is  defined  to  be  as  follows: 

Explosive  ordnance  which  has  been  {nimed,  fuzed,  armed,  or  otherwise  prqiaied  for  action, 
and  which  has  been  launched,  placed,  fired,  or  released  in  such  a  manner  as  to  constitute  a 
hazard  to  friendly  operations,  installations,  personnel,  or  materiel  and  remains  unexploded 
either  through  m^function  or  design  or  for  any  other  cause. 

2  “Statement  of  Research  Need,”  Strategic  Enviroiunent  Research  and  Development  Program,  1996, 
downloaded  from  the  Internet,  copy  on  file  with  IDA. 
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With  the  drawdown  of  U.S.  military  forces  beginning  in  1985  and  the  transfer  of 
surplus  military  facilities  to  other  governmental  agencies,  educational  institutions,  or  their 
outright  sale,  the  potential  for  civilian  contact  with  UXO  rose  dramatically.^  As  a  result  of 
The  BRAC  process  and  recent  amendments  to  statute  dealing  with  community  redevelop¬ 
ment  and  the  transfer  of  formerly  used  Defense  sites  (FTJDS),  there  is  even  greater 
emphasis  on  speedy  conversion  of  DoD  facilities  to  alternative  uses  under  the  control  of 
non-DoD  govenunent  agencies  or  even  private  owners.  The  net  effect  of  these  efforts  is  to 
increase  the  possibility  of  aUowing  untrained  civilians  access  to  lands  potentially 
contaminated  with  UXO.'* 

DoD  is  concerned  about  the  health  and  safety  of  its  civilian  and  military  employees 
as  well  as  its  contractors.  Peacetime  clearance  of  UXO  from  active,  reserve,  and  inactive 
weapons  ranges  poses  significant  safety  challenges  for  DoD  and  contractor  personnel. 
Clearance  of  UXO  from  DoD  facilities  slated  for  transfer  to  non-DoD  organizations  raises 
even  more  demanding  safety  challenges  than  UXO  clearance  from  ranges  or  DoD  installa¬ 
tion  rehabilitation/environmental  remediation.  UXO  clearance  at  facilities  scheduled  for 
conversion  is  especially  demanding  because  of  the  possibility  of  post-transfer  incidents 
involving  detonation  of  hitherto  unknown  UXO  resulting  in  injuiy  and  damage  to  non-DoD 
personnel  and  materiel.  The  possibility  of  such  incidents  requires  further  UXO  clearance 
efforts  by  DoD  personnel  at  FUDS.  They  also  represent  both  the  greatest  safety  challenge 
to  DoD  personnel  and  greatest  financial  liabilities  to  DoD.  The  extraordinary  safety 
problems  arise  after  transfer  of  DoD  lands  to  others  because  of  multiple  uses  of  land  and 
the  commingling  of  civilians  and  non-DoD  government  personnel  with  no  knowledge  or 
concern  for  munitions  safety.  The  high  financial  liability  to  DoD  results  from  (1)  the  high 
costs  of  UXO  clearance  after  land  transfer  has  occurred  and  (2)  the  statutory  liability  of 
DoD  to  pay  damages  to  victims  of  UXO  incidents  which  occur  after  DoD  has  certified  that 
its  facilities  or  lands  are  cleared  of  UXO  or  hazardous  ordnance  waste.  ^ 


^  Hie  Defense  Authorization  Amendments  and  Base  Closure  and  Realignment  Act  of  1988  (BCRA  88, 
Tide  II  of  Pub.  L.  100-526,  10  U.S.C.  §2687  note),  and  the  Defense  Base  Closure  and  Realignment 
Act  of  1990  (DBCRA  90.  Part  A  of  Tide  XXDC  of  Pub.  L.  101-510,  10  U.S.C.  §2687  note), 
establish  the  basic  requirements  for  identifying  and  implementing  domestic  military  base  closures 
and  realignments. 

'*  See  Army  and  Air  Force  Regulations  on  the  Management  of  Real  Property;  see  also  Base  Closure 
Manual  which  discusses  implementation  of  statutes  intended  to  give  the  pubUc  greater  access  to  DoD- 
owned  lands,  even  if  DoD  determines  that  lands  must  be  retained  in  “reserve”  status.  See  also  the 
Pressed  Rule  for  transfer  of  closed  DoD  wdnance  ranges.  All  of  these  items  were  downloaded  from 
the  Internet  and  are  on  rile  at  IDA. 

^  Resource  Conservation  and  Recovery  Act  (RCRA),  42  U.S.C.  §6901  el  seq.\  40  CFR  Parts  240-281. 
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As  DoD  has  gained  more  knowledge  and  has  begun  to  comply  fully  with  the  law, 
the  costs  of  facilities  cleanup  and  remediating  environmental  damage  have  soared.  A  1995 
Congressional  Budget  Office  study  summarized  the  financial  projections  and  impact  of 
UXO  and  other  environmental  cleanup  efforts  within  DoD  in  the  following  terms: 

Changes  in  DoD’s  cost  estimates  and  budget  plans  continue  to  reveal  the 
high  degree  of  uncertainty  that  characterizes  the  cleanup  program.  In  1985 
DoD  estimated  that  completing  the  cleanup  program  would  cost  between 
$6.9  billion  and  $13.7  billion  (1995  dollars).  DoD  recently  estimated  that 
the  program  could  cost  about  $30  billion.  Annual  budget  requests  have  also 
risen,  1989,  the  department  estimated  that  it  would  need  between 
$9(X)  million  and  $1.2  billion  to  fund  cleanup  requirements  in  1994;  the 
Congress  authorized  about  twice  the  higher  estimate.  Similar  trends  in  cost 
growth  have  occurred  at  individual  military  bases. 

The  Inspector  General  of  DoD  found  that  average  cleanup  costs  for  defense 
facilities  scheduled  to  be  closed  were  about  60  percent  higher  than  initial 
estimates.^ 

The  focus  of  DoD’s  research,  development,  test,  and  evaluation  (RDT«&E)  efforts 
in  UXO  detection,  classification,  identification,  and  remediation  has  shifted  in  response  to 
changes  in  the  program’s  technical,  economic,  and  political  environment  Improving 
personal  safety  for  individuals  involved  in  environmental  remediation  efforts  at  DoD  facili¬ 
ties  has  become  a  major  research  and  development  (R&D)  need.  The  need  to  be  confident 
in  the  ability  to  clear  UXO  from  lands  to  be  released  by  DoD  for  other  purposes  as  a  result 
of  the  drawdown  of  DoD  infrastructure  has  also  driven  UXO  remediation  R&D  require¬ 
ments.  The  high  current  and  potential  long-term  costs  of  UXO  clearance  and  liability  for 
post-transfer  incidents  has  increased  emphasis  on  development  of  technologies  and 
methods  for  rapid,  reliable,  and  accurate  UXO  detection,  classification,  identification,  and 
removal,  especially  for  buried  UXO. 


^  Cleaning  Up  Defense  Installations:  Issues  And  Options,  Congressional  Budget  Office,  Washington 
D.C.,  January  1995).  ’ 
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There  has  been  an  increase  in  high-level  attention  within  the  Department  of 
Defense"^  and  also  by  the  General  Accounting  Office*  (GAO)  and  the  Congressional  Budget 
Office  9  which  has  also  helped  to  refocus  RDT&E  efforts. 

The  Department  of  Defense  has  developed  the  Environmental  Security  Technology 
Certification  Program  (ESTCP).  The  ESTCP  seeks  to  identify  those  technologies  which 
would  provide  substantial  return  on  investment  through  cost  savings  and  improved 
efficiency.  ESTCP  seeks  to  move  aggressively  from  field  trials  to  acquisition,  where 
demonstrations  provide  evidence  of  cost-effective  performance  and  market  potential.*® 
This  program  demonstrates  and  validates  the  most  promising  innovative  technologies  that 
address  DoD’s  most  urgent  environmental  needs  and  are  projected  to  pay  back  the  invest¬ 
ment  within  5  years  through  cost  savings  and  improved  efficiencies.  The  ESTCP  is  spon¬ 
soring  a  series  of  demonstrations  of  UXO  detection  systems  as  part  of  its  broad  technology 
demonstration  program. 

A  1995  review  of  the  Defense  Environmental  Program  Management  found  that 
current  UXO  detection  technology  was  very  costly  and  could  not  meet  DoD  requirements 
because  of  an  inability  to  detect  ordnance  buried  below  3  feet.  The  potential  for  migration 
of  deep  buried  ordnance  to  the  surface  is  a  major  problem.  DoD  is  committed  to  cleaning 
up  active  bases,  FUDS,  and  bases  subject  to  realignment  and  closure  in  a  manner  that 
allows  maximum  public  use  and  conversion  to  other  governmental  or  private  use.  Unless 
such  ordnance  can  be  detected  and  cleared  prior  to  DoD  release  of  lands  for  other  purposes, 
DoD  retains  long-term  clearance  responsibility  and  potential  financial  liability. 

GAO  reported  three  significant  barriers  to  the  use  of  new  technology  for  containing 
and  cleaning  up  contamination  at  DoD  facilities: 


I  See,  for  example,  Report  of  the  Defaise  Science  Board  Task  Force  on  Environmental  Security 
dated  22  April  1995,  downloaded  from  the  Internet,  copy  on  fde  at  IDA. 

*  See  for  example,  the  following  GAO  Reports  to  Congress: 

Environmental  Cleanup:  Inconsistent  Sharing  Arrangements  May  Increase  Defense  Costs 
(Washington,  D.C.:  GAO/NSIAD-94-231,  July  1994);  Military  Bases:  Environmental 

Impact  at  Closing  Installations  (Washington,  DC:  GAO/NSIAD-95-70,  February  1995); 
Environmental  Protection:  Status  of  Defense  Initiatives  for  Cleanup,  Compliance  and 
Technology  (Washington,  D.C.:  GAO/NSIAD-96-155,  August  1996). 

9  Congressional  Budget  Office,  Cleaning  Up  Defense  Installations:  Issues  and  Options  (Washington, 
D.C.:  CBO,  January  1995). 

10  See  “[DoD]  Environmental  Security  Technology  Certification  Progi^  Description,”  downloaded  from 
the  Internet,  30  September  1996,  copy  on  file  with  IDA.  Available!  http!//estcp.xservices.coin/ 
geninfi^docs/guide-l  1  .doc. 
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•  Conflicting  priorities  prevent  the  approval  of  innovative  approaches  for 
cleanup. 

•  Field  officials  may  associate  the  newer  technologies  with  unacceptable  levels 
of  risk. 

•  On-site  contractors  may  favor  particular  technologies  on  the  basis  of  their  own 
experience  and  investments. 

The  ESTCP  asked  IDA  to  assist  the  program  by  assessing  measures  of  effective¬ 
ness  (MOEs)  and  UXO-detection  performance  descriptions  that  would  be  useful  to  DoD 
managers  responsible  for  ordnance  test  and  training  range  clearing  operations  as  well  as 
UXO  contamination  cleanup  at  Installation  Restoration  Program,  BRAC,  and  FUDS  loca¬ 
tions.  This  report  describes  in  general  terms  the  current  procedures  used  for  peacetime 
clearing  of  UXO  from  CONUS  DoD  test  and  training  ranges.  Installation  Restoration 
Program  sites,  BRAC  facilities,  and  FUDS.  This  report  has  two  goals:  (1)  to  present  a 
set  of  MOEs  for  UXO  technology  demonstration,  and  (2)  to  describe  the  methods  of  data 
collection  and  analysis  necessary  and  sufricient  to  determine  whether  the  technologies 
included  in  the  UXO  demonstration/validation  program  offer  sufficient  promise  to  warrant 
further  investment.  The  report  relies  upon  the  written  record  of  past  UXO  technology 
development  efforts,  DoD  regulations  and  administrative  practices,  as  well  as  interviews 
with  Explosive  Ordnance  Disposal  (EOD)  personnel  to  capture  as  much  information  as 
possible  to  meet  the  needs  of  ESTCP  managers  in  developing  and  implementing  realistic 
demonstration/validation  experiments. 

Recent  DoD  UXO  R&D  efforts  have  focused  on  improving  sensors  for  use  in 
the  detection  of  UXO  during  initial  site  surveys  and  in  discrimination  of  UXO  from 
other  subsurface  objects  found  during  detailed  site  investigation.  Improving  performance 
in  these  areas  offers  an  opportunity  to  use  more  efficiently  limited  UXO  remediation 
resources,  including  UXO  qualified  military  personnel  (EOD)  specialists  and  civilian 
munitions  destroyers. 

Tests  of  sensors  can  be  designed  to  produce  data  that  either  investigate  the 
phenomenology  of  the  system  or  that  determine  the  utility  of  the  system  in  real-world 
conditions.  In  either  case,  data  must  be  taken  in  such  a  way  as  to  support  clearly  articulated 


David  R.  Warren,  “Environmental  Protection:  Challenges  in  Defense  Environmental  Program 
Management,”  Testimony  befcme  the  Subconunittee  on  Military  Readiness  and  Military  Tnstallatinn«t 
and  Facilities,  Committee  on  National  Security,  House  of  Representatives  (Washington,  D.C.: 
GAO/r-NSIAD-95-121,  March  24, 1995),  pp.  9-10. 
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measures  of  performance.  Definitions,  standard  scoring,  and  report  protocols  are  neces¬ 
sary  to  make  the  results  of  the  demonstration  easily  understood  by  the  entire  community. 
In  this  same  vein,  the  data  must  demonstrate  convincingly  to  the  user  that  the  technology  is 
worthwhile.  EOD-qualified  personnel  and  the  users  of  any  system  that  is  ultimately 
developed  (EOD-qualified  DoD  and  contractor  personnel)  must  feel  comfortable  with  the 
safety  and  reliability  of  a  system  if  it  is  to  see  widespread  use.  A  technology  should  not 
require  a  drastic  departure  from  current  UXO  clearance  protocols,  which  are  designed  to 
protect  the  safety  of  personnel,  without  good  reason. 

This  report  focuses  on  sensor  technologies.  We  will  attempt  to  inject  our  knowl¬ 
edge  and  understanding  of  UXO  detection  technology,  current  procedures  for  remediation 
of  UXO,  the  views  and  concerns  of  EOD  personnel,  and  lessons  learned  from  past  tech¬ 
nology  demonstrations  to  guide  commentary  on  the  development  of  techniques  to  record, 
analyze,  and  archive  data  to  support  both  the  UXO  technology  development  and  operational 
communities.  It  is  our  hope  that  this  analysis  contribute  to  the  reduction  of  barriers  to  the 
adoption  of  new  technology  noted  by  GAO. 
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II.  UXO  REMEDIATION:  PROCESSES  AND  BEHAVIORS 


A.  OVERVIEW 

The  Department  of  Defense  is  responsible  for  the  remediation  of  health,  safety,  and 
actual  or  potential  adverse  environmental  effects  of  UXO  located  on  Military  Ranges,  DoD 
facilities  being  disposed  of  as  a  result  of  BRAG  Commission  actions  or  service  excess 
property  disposal  processes,  and  FUDS^  on  which  UXO  has  been  discovered.  Remedia¬ 
tion  efforts  must  be  carried  out  in  accordance  with  processes  mandated  by  statute  and 
amplified  by  Code  of  Federal  Regulations,  DoD  Issuances,  Service  guidance,  and  facility- 
specific  guidance.  Remediation  efforts  may  also  be  described  in  behavioral  terms — ^what 
actually  happens  on  the  ground.  In  this  chapter,  we  summarize  the  processes  mandated  by 
law  and  regulations,  describe  the  personnel  qualifications  and  training  of  individuals  who 
actually  clear  UXO  from  the  field,  and  report  on  the  “real-world”  processes  as  described  to 
us  by  participants  in  UXO  clearing  operations. 

B .  RANGE  CLEARANCE  PROCESSES 

The  Department  of  Defense  maintains  a  number  of  Military  Ranges  defined  as  “any 
land  mass,  or  water  body  that  is  or  was  used  for  the  purpose  of  training,  research,  develop¬ 
ment,  testing,  or  evaluation  of  Military  Munitions.”^  Ranges  fall  into  five  categories,  as 
follows: 

1.  Active  (used  for  training,  research,  development,  testing,  or  evaluation  of 
military  munitions). 

2.  Inactive  (range  not  currently  in  use  but  still  considered  to  be  a  potential  range 
area). 


^  According  to  the  U.S.  Anny  Corps  of  Engineers,  a  FUDS  is  “a  Site  that  has  been  owned  by,  leased  to, 
possessed  by,  o'  otherwise  under  the  jurisdiction  of  DoD.  The  FUDS  program  does  not  apply  to  those 
sites  outside  the  U.S.  jurisdiction.”  For  further  information  see  the  FUDS  page  at  the  Defense 
Envirotunental  Restoration  Program  World  Wide  Web  Site,  available:  http://dogberLncr.usace.army. 
mil/military/detp/fuds/fuds.btm. 

2  See  Paragr^h  IV.B.2,  “Military  Range,”  in  “Section  by  Section  Analysis,  Proposed  Range  Rule,” 
found  at  Note  #78,  DENDC  Data  Manager,  12:24  p.m.,  September  27,  1996,  available: 
http://denix.cecer.armyujnil/denix/Webnotes;  copy  on  file  at  IDA. 
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3.  Transferred  (a  military  range  that  has  been  released  from  military  control,  i.e., 
FUDS). 

4.  Transferring  (a  military  range  that  has  been  identified  for  closure  and  transfer 
as  part  of  the  BRAC  process  or  has  been  declared  surplus  real  property  by  a 
military  Service  pursuant  to  its  own  processes  for  disposition  of  unneeded  real 
property) 

5.  Closed  (range  taken  out  of  service  by  the  military  and  put  to  new  uses 
incompatible  with  further  use  as  a  range  but  still  under  the  control  of  the 
Department  of  Defense). 

Significant  opportunities  exist  for  UXO  contamination  in  each  type  of  military  range 
called  out  above.  DoD  is  committed  to  reducing  safety  risks  associated  with  exposure  of 
the  public  and  clearance  personnel  to  military  munitions.  The  approach  to  clearing  UXO 
from  the  ranges  varies  somewhat  with  the  type  of  range  and  intent  for  future  use. 

Active  ranges  are  usually  cleared  of  UXO  by  either  active  duty  UXO-qualified 
military  personnel  (EOD  specialists)  assigned  to  the  range  or  UXO-qualifled  civihan 
contractors  (munitions  destroyers )3  under  contract  to  the  facility  or  to  RDT&E  programs 
utili2dng  the  range.  Clearance  activities  are  part  of  the  day-to-day  operation  of  the  range. 
The  frequency  of  clearance  operations  is  based  on  guidance  issued  by  the  facility 
commander  in  conjunction  with  broad  direction  from  the  DoD  Explosives  Safety  Board.'* 

Clearance  of  UXO  may  also  be  undertaken  in  conjunction  with  Installation  Restora¬ 
tion  Programs  (IRP)  designed  to  mitigate  past  practices  at  DoD  sites  which  may  have 
contributed  to  the  release  of  pollutants  into  the  environment.  IRP  efforts  are  conducted  in 
accordance  with  rules  and  procedures  established  by  DoD  under  the  Comprehensive 
Environmental  Response,  Compensation,  and  Liability  Act  (CERCLA)  as  amended  by 


^  “Section  3.d.  UXO  Qualified  Personnel,”  in  “Safety  Concepts  and  Basic  Considerations  for  Unexploded 
Ordnance  (UXO)  Operations:  Interim  Guidance,”  Ordnance  and  Explosives  Center  of  Excellence,  U.S. 
Army  Corps  of  Engineers,  Huntsville  Division,  Revised  16  February  1996,  available: 
http://w2.hnd.usace.army.mil/oew/policy/safeconl.htm,  and  on  file  at  EDA,  deOnes  UXO  Qualified 
Personnel  as  follows: 

...U.S.  citizens  who  have  graduated  from  the  U.S.  Army  Bomb  Disposal  School,  Aberdeen, 
Maryland,  or  the  U.S.  Naval  Explosive  Ordnance  Disposal  (EOD)  School,  Indian  Head,  MD. 
Graduates  of  the  EOD  Assistant  Course,  Redstone  Arsenal,  AL,  or  Eglin  AFB,  FL,  with  more 
than  3  years  combined  active  duty  nulitary  EOD  and  contractor  UXO  experience  shall  also  be 
UXO  qualified. 

'*  See  “Paragr^  F.  Functions,”  in  DoD  Directive  6055.9,  The  DoD  Explosives  Safety  Board, 
25  November  1983,  downloaded  from  DefenseLink  Publications,  DoD  Issuances,  available:  http:// 
www.dtic.mil/defenslink;  on  file  at  IDA. 
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Superfund  Amendments  and  Reauthorization  Act  (SARA)  requirements  and  Resource 
Conservation  Recovery  Act  (RCRA)  as  outlined  below. 

It  is  important  to  bear  in  mind  that  Military  Ranges  are  usually  used  for  multiple 
RDT&E  and  training  purposes.  Ranges  are  separated  by  function,  not  ordnance  type. 
Although  RDT&E  ranges  are  usually  separate  from  training  and  maneuver  ranges,  multiple 
types  of  munitions  within  a  family  of  ordnance  (e.g.,  artillery)  are  expended  on  each  range. 
Furthermore,  oftentimes  multiple  families  of  ordnance  have  been  expended  at  any  given 
range.  It  is  not  uncommon  to  find  UXO  remnants  from  artillery,  surface-to-surface 
rockets,  air-to-surface  rockets  and  missiles,  mortars,  and  bombs  scattered  on  the  same 
range.  Range  clearance  (for  all  types  of  ranges)  implies  mixed-ordnance  contamination. 

C.  STATUTORY  ENVIRONMENTAL  RESTORATION  PROGRAMS 

Several  different  statutes  compel  the  DoD  to  undertake  significant  environmental 
restoration  activities  in  which  discovery,  rendering  safe,  and  disposal  of  UXO  figure 
prominently.  Key  statutory  processes  or  specific  laws  are  the  following:  ^ 

•  Defense  BRAC  processes  which  are  established  by  several  statutes.  BRAC 
requires  DoD  compliance  with  National  Environmental  Policy  Act  and  other 
environmental  laws  identified  below. 

•  National  Environmental  Policy  Act  (NEPA),  42  U.S.C.  §4321  et  seq.  NEPA 
requires  that  DoD  Components  analyze  potential  environmental  impacts  of 
proposed  actions  and  alternatives  for  base  disposal  decisions.  Implementing 
regulations  (40  CFR  Parts  1500-1508)  describe  in  detail  the  specific  parameters 
which  DoD  managers  must  include  in  their  analyses  of  potential  environmental 
impacts  as  well  as  processes  by  which  such  parameters  are  evaluated. 
Executive  Order  11514  as  amended  by  Executive  Order  11991  provides 
additional  guidance  to  DoD. 


^  The  BRAC  process  is  set  forth  in  a  dynamic  set  of  statutes  which  include  but  are  not  necessarily 
limited  to  the  following: 

•  Defense  Authorization  Amendments  and  Base  Closure  and  Realignment  Act  of  1988  (BRCA 
88)  Pub.  L.  100-526,  10  U.S.C.  §2687  note,  24  October  1988 

•  Defense  Base  Closure  and  Realignment  Act  of  1990  (DBRCA90),  Pub.  L.  101-510, 
10  U.S.C.  §2687  note,  5  November  1990 

•  National  Defense  Authorization  Act  for  Fiscal  Years  1992  and  1993  (NDAA  92/93),  Pub.  L. 
102-190,  §§334(a),  2821, 2827, 5  December  1991 

•  National  Defense  Authorization  Act  for  Fiscal  Year  1993  (NDAA  93),  Pub  L.  102-484, 
23  October  1992 

See  Appendices  A,  B,  and  C  to  DoD  Base  Reuse  Implementation  Manual  (Washington,  DC: 
Department  of  Defense,  July  1995),  avaUable:  http://www.acq.osd.mil/i^reinvest/manual/ 
manual.btml  and  on  file  at  IDA. 
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•  Comprehensive  Environmental  Response,  Compensation,  and  Liability  Act 
(CERCLA),  42  U.S.C.  §9601  et  seq.  CERCLA  requires  DoD  to  undertake  any 
needed  response  action  to  clean  up  contamination  found  at  sites  formerly  used 
by  DoD  activities.  Such  response  measures  must  address  risks  to  human  health 
and  to  the  environment  posed  by  past  releases  of  hazardous  substances. 
Section  120(h)  of  this  act  amended  by  the  Community  Environmental  Response 
Facilitation  Act  (CERFA)  governs  the  identification  of  uncontaminated  parcels 
and  covenant  requirements  for  deed  transfers  of  contaminated  parcels.  40  CFR 
Parts  300-311  provide  additional  guidance  with  respect  to  both  process  and 
content  of  DoD  environmental  restoration  activities  to  address  contamination 
resulting  from  past  actions. 

•  Resource  Conservation  and  Recovery  Act  (RCRA),  42  U.S.C.  §6901  et  seq. 
RCRA  requires  DoD  to  establish  a  management  system  for  hazardous  waste  and 
provides  corrective  action  authority  for  cleanup  of  solid  waste  management 
units.  40  CFR  Parts  240-281  provide  further  implementing  direction. 

•  Department  of  Defense  Environmental  Restoration  Program  (DERP), 
10  U.S.C.  §2701-2708  and  2810  (enacted  as  secUon  211  of  Superfund 
Amendments  and  Reauthorization  Act  of  1986).  DERP  provides  authority  for 
DoD  to  remediate  environmental  hazards  emanating  from  active  DoD  sites 
which  may  be  posing  potential  risk  to  the  health,  safety,  and  environment  of 
surrounding  communities.  Further  implementing  guidance  is  contained  within 
DoD  Instruction  4715.7,  “Environmental  Restoration  Program,”  dated  April  22, 
1996. 

These  statutes  and  their  implementing  Federal  Regulations,  Presidential  Executive 
Orders,  and  DoD  Issuances  form  a  common  procedural  framework  within  which  the  field 
work  necessary  to  find,  render  safe,  and  dispose  UXO  takes  place.  The  following  process 
is  generally  followed  whenever  a  DoD  facility  is  proposed  for  closure  or  realignment 
(BRAC  processes),  UXO  is  discovered  at  a  FUDS,  or  DoD  has  determined  it  is  necessary 
to  remedy  an  environmental  problem  at  an  active  DoD  facility  or  site  (IRP). 

The  discussion  of  generic  DoD  environmental  remediation  process  that  follows 
assumes  that  there  is  no  immediate  risk  to  public  health  or  safety.  A  different  process  is 
used  when  such  risks  do  arise.  Although  the  form  and  detailed  documentation  generated  in 
the  process  of  preparing  to  remedy  UXO  contamination  at  a  DoD  site  depends  upon  which 
statutory  or  regulatory  provision  governs  the  effort,  the  generic  process  described  in  Table 
2  is  quite  similar.  In  many  instances,  the  broader  environmental  hazard  assessments 
undertaken  pursuant  to  the  CERCLA  requirements  find  historical  evidence  of  ordnance 
activities  at  a  site,  which  in  turn  prompts  more  thorough  records  and  physical  inspections. 
Table  2  lists  the  major  steps  and  the  purpose  of  each  in  a  site  remediation  effort 
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Table  2.  Generic  DoD  Environmental  Remediation  Process* 


Task 

Purpose 

Preliminary  Assessment 

Installation-wide  study  to  determine  if  sites  are  present  that  pose  hazards  to 
public  health  or  the  environment.  Available  information  is  collected  on  the 
source,  nature,  extent,  and  magnitude  of  actual  and  potential  hazardous 
substance  releases  at  sites  on  the  installation. 

Site  Inspection 

Sampling  and  analysis  to  determine  actual  existence  of  actual  site  contamina¬ 
tion.  Information  used  to  evaluate  site  condition  and  determine  response  action 
needed. 

Remedial  Investigation 

Additional  site  investigations,  sampling,  and  anaiytical  studies  and  activKies  to 
determine  the  nature,  extent,  and  significance  of  contamination;  focus  is 
determining  risk  to  the  general  population  posed  by  contamination. 

Feasibility  Study 

Conducted  concurrently  with  Remedial  Investigations;  evaluation  of  remedial 
action  alternatives  to  determine  which  alternative  would  provide  the  required 
protection  against  future  hazards. 

Remedial  Design  and/or 
Removal  Design 

Detailed  plans  for  chosen  remedial/removal  actions  prepared,  including  plans 
for  removal  and  off-site  disposal  of  UXO  if  in  situ  destruction  is  not  feasible. 

Remedial  Action  and/or 
Removal  Action 

Implementation  of  remedial/removal  plans  including  in  situ  destruction  of  UXO 
and/or  removal  of  UXO  and  associated  UXO  waste. 

Interim  Remedial  and/or 
Removal  Action 

Remedial/Removal  Action  taken  at  any  time  during  the  cleanup  process  to 
protect  public  health  and/or  to  control  contaminant  releases  to  the  environment. 

Remedy  in  Place; 
Functioning  as  Intended 

Remedial  action  is  functioning  properly  and  performing  as  designed. 

Remediation  includes  such  actions  as  the  operation  of  pump-and-treat  systems 
that  will  take  decades  to  complete  cleanup. 

*  This  table  is  modeled  after  Appendbc  I  in  Militafy  Bases  Environmental  Impact  at  Closing  Installations  (Washington,  DC; 
U.S.  General  Accounting  Office,  February  1995). 


Depending  upon  the  nature  of  the  threat  to  health,  safety,  and  the  environment,  the 
generic  process  outlined  above  is  subject  to  specific  amendment  in  accordance  with 
Environmental  Protection  Agency  guidance.  Emergency  Removal  Actions  address 
immediate,  unacceptable  hazards.  Time-Critical  Removal  Actions  respond  to  releases  of 
contaminants  into  the  environment  requiring  action  in  less  than  6  months.  Non-Time- 
Critical  Removal  Actions  respond  to  releases  or  potential  releases  of  contaminants  that  can 
start  more  than  6  months  after  a  determination  that  a  response  is  needed.  A  decision  on 
whether  to  conduct  a  Time-Critical  Removal  Action,  a  Non-Time-Critical  Removal  Action, 
or  a  combination  of  both  is  usually  made  at  the  completion  of  the  Preliminary  Assessment 
or  the  Site  Evaluation  Phase  of  the  environmental  remediation  process.^ 


®  See  U.S.  Army  Corps  of  Engineers,  Huntsville  Division,  “Removal  Action  Planning  for  Ordnance 
and  Explosives  Sites:  Procedural  Document,”  CEHND  1115-3-524,  January  1995,  Section  7.1.1, 
available:  http://w2Jmd.usace.army.mil/oew/policy/cerctxt2.btml  and  on  file  at  IDA. 


n-5 


D.  CLEARING  UXO:  SAFETY  FIRST 


DoD  has  summarized  the  technical  challenge  it  faces  in  cleaning  up  UXO  from 
military  ranges  and  other  current  or  former  DoD  facilities: 

Military  Mimidons  are  designed  to  injure  or  kill  people,  and/or  to  damage  or 
destroy  property.  As  such.  Military  Munitions  are  unlike  any  other  product 
for  which  an  envirorunentaUy  based  response  might  be  otherwise  appro¬ 
priate.  During  any  response  activity,  the  presence  or  suspected  presence  of 
Military  Munitions  creates  unique  explosives  safety  challenges.  Before 
undertaking  any  response  action... DoD  must  first  consider  the  explosives 
safety  risks  inherent  in  locating,  investigating,  evaluating,  and  responding 
to... areas  where  Military  Munitions  are  known  or  suspected  to  be  present. 

The  explosive  safety  risk  is  equally  great  regardless  of  whether  Militaty 
Munitions  (including  UXO)  or  Other  Constituents  are  being  addressed  in 
the  response  action.  Response  personnel,  even  those  specially  trained  to 
deal  with  the  explosives  safety  hazards  associated  with  Military  Munitions, 
must  not  be  exposed  to  an  unreasonable  explosive  safety  risk  in  order  to 
address  less  compelling  environmental  concerns.  The  potential  explosives 
safety  risk  to  response  personnel  increases  as  the  density  of  Military  Muni¬ 
tions,  e.g.,  UXO,  increases.  Additionally,  rough  terrain  and  thick  vegeta¬ 
tion  restrict  visibility  and  mobility,  thereby  substantially  increasing  the 
explosives  safety  hazards  associated  with  response  activities.  Response 
activities  are  made  more  difficult  and  dangerous  because  technology  is  not 
yet  sophisticated  enough  to  ensure  positive  detection,  identification,  and 
subsequent  removal  of  all  Military  Munitions  in  any  given  area."^ 

The  Army  Corps  of  Engineers,  on  behalf  of  the  Secretary  of  the  Army,  is  the  DoD 
Executive  Agent  for  the  remediation  of  environmental  problems  at  FUDS  and  acts  as  the 
lead  agency  in  conducting  restoration  activities  at  eligible  properties  on  behalf  of  DoD 
Components.  It  also  acts  as  the  lead  agency  in  environmental  restoration  activities  at 
operational  installations,  closing  and  realigning  installations,  and  FUDS.*  The  Huntsville 
District,  which  hosts  the  Mandatory  Center  of  Expertise  (OECX),  observes  in  its  basic 
guidance  that  “there  is  no  ‘safe’  procedure  for  dealing  with  UXO,  merely  procedures  which 
are  considered  least  dangerous.”^ 


^  “IV.  Section-by-Section  Analysis  Paragr^  C,  Department  of  Defense  Proposed  Range  Rule,” 
dated  27  September  1996,  available:  http://denix.cecer.army.mil/denix/Webnotes;  copy  on  file  at  EDA. 

*  DoD  Instruction  4715.7,  “Environmental  Restoration  Program,”  dated  22  Aiwil  1996,  available: 
http://www.dtic.inil/defenselink/publications;  copy  on  file  at  IDA.  Each  service  is  responsible  for 
clearing  its  own  active  ranges  and  clearing  UXO. 

®  U.S.  Army  Corps  of  Engineers  Huntsville  Center,  “OE  CX:  Interim  Guidance  Safety  Concepts 
and  Basic  Consi^rations  for  Unexploded  Ordnance  (UXO)  Operations,”  Revised  16  February  1996, 
available:  http://w2.hnd.usace.anny .mil/oew^licy/safeconl.html;  copy  on  file  at  IDA. 
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E.  UXO  CLEARANCE:  FIELD  PRACTICES 


UXO  is  generally  cleared  by  EOD  specialists.*®  The  following  is  an  abbreviated 
list  of  the  duties  required  of  EOD  technicians: 

1.  Detect  explosive  ordnance  on  the  surface  and  subsurface  (land  only). 

2.  Identify  explosive  ordnance. 

3.  Render  safe  unexploded  explosive  ordnance. 

4.  Recover  explosive  ordnance. 

5.  Evacuate  explosive  ordnance. 

6.  Dispose  of  explosive  ordnance. 

7.  Disassemble  and  render  inert  foreign  explosive  ordnance  for  technical  intel¬ 
ligence. 

8.  Disseminate  technical  information  on  enemy  explosive  ordnance  materials. 

Although  this  list  is  usually  applied  to  problems  in  a  wartime  environment,  it  forms 
the  core  focus  of  peacetime  training.  Use  of  EOD  personnel  to  clear  UXO  on  Military 
Ranges  and  to  provide  a  range  of  emergency  and  time-critical  removal  action  responses  is 
also  viewed  as  “training.”  Formally  trained  active  duty  and  former  military  EOD  specialists 
form  the  cadre  of  UXO-qualified  personnel. 

Before  UXO-qualified  persoimel  begin  the  process  of  physically  clearing  UXO 
from  a  site,  a  great  deal  of  paperwork,  study,  and  analysis  is  undertaken.  The  overall 
process  outlined  above  in  generic  terms  oftentimes  begins  with  the  discovery  of  UXO  on  or 
near  the  land  surface.  The  process  goes  through  a  set  of  formal  phases  outlined  below, 
culminating  in  either  the  destruction  of  the  UXO  in  place  or  its  removal  for  subsequent 
controlled  destruction. 

1 .  Phase  0:  Discovery  of  UXO 

The  discovery  of  UXO  comes  about  in  many  different  ways.  Table  3  summarizes 
the  environments  and  common  methods  by  which  UXO  is  discovered. 


*  ®  For  additional  detailed  information  see  the  following  documents; 

AR  611-105,  Selection,  Processing,  and  Training  of  Officer  Volunteers  for  Explosive 
Ordnance  Disposal  Duty,  3  September  1985;  ARTEP  9-527-30-MTP,  Mission  Training  Plan 
pr  Ordnance  Detachment,  Explosive  Ordnance  Disposal,  2  September  1993;  ARTEP  9-527- 
MTP,  Mission  Training  Plan  for  Ordnance  Detachment,  Explosive  Ordnance  Disposal  Control 
Team,  8  September  1993. 
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Table  3.  UXO  Discovery  and  initial  Response 
(Peacetime  Occurrences,  CONUS) 


Site  Type 

Discoverer 

Jurisdiction 

Emergency 

Response 

Long-Term  DoD 
Response 

Active  Military 

Range 

Active  Duty  Military, 
Reserve  Military, 
Contractor, 
Dependent;  Civilian 
Who  Has  Made 
Unlawful  Entry  Onto 
Military  Range 

Military  Service 
Responsible  for 
Range 

First  Available  EOD 
Unit  if  Emergency; 
Parent  Service  EOD 
if  Not  an  Emergency 

Parent  Service 

Inactive  Military 
Range;  DoD  Facility 

Active  Duty  Military, 
Reserve  Military, 
Contractor, 
Dependent; 
Authorized  Civilian; 
Unauthorized 

Civilian 

Military  Service 
Responsible  for 
Range;  Facility 

First  Available  EOD 
Unit  if  Emergency; 
Parent  Service  EOD 
if  Not  an  Emergency 

Parent  Service 

Transferring  Military 
Range;  Transferring 
Military  Facility 
within  CONUS 
(BRAC) 

Active  Duty  Military, 
Reserve  Military, 
Contractor. 
Dependent; 
Authorized  Civilian; 
Unauthorized 

Civilian 

U.S.  Army 

Contractor 
Responsible  for 
Environmental 
Cleanup;  52nd  EOD 
Group  per  MOA 

Parent  Military 
Service;  U.S.  Army 

Transferred  Military 
Range;  Transferred 
DoD  Facility  (FUDS) 

DoD  Contractor; 
Authorized  Civilian; 
Unauthorized 

Civilian 

Non-DoD  Federal 
Agency  or 
State/Local  Civil 
Emergency 
Authorities 

State  or  Local  Law 
Enforcement  EOD; 
U.S.  Army  upon 
request;  First 
available  EOD  if  an 
emergency 

U.S.  Army  as 
Executive  Agent  for 
the  Department  of 
Defense 

The  Corps  of  Engineers,  Huntsville  Division  (CEHND),  has  established  an  OECX 
within  the  Huntsville  District  responsible  for  providing  consolidated  guidance  to  all  DoD 
Components  on  the  proper  handling  of  UXO.  It  is  also  responsible  for  coordinating 
emergency  and  nonemergency  EOD  unit  responses  among  military  units  and  civil  law 
enforcement  authorities  where  assistance  is  requested.  Since  it  is  often  difficult  to  deter¬ 
mine  whether  a  newly  discovered  UXO  item  constitutes  an  imminent  danger  to  the  health 
and  safety  of  persons  or  property,  discovery  is  often  treated  as  an  emergency. 

Figure  1  summarizes  common  decision  routes  by  which  responsibility  for  emer¬ 
gency  response  to  initial  discovery  of  UXO  is  allocated  among  local,  state,  and  DoD 
authorities. 
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Figure  1.  EOD  Response  Coordination''^ 

Calls  to  the  CEHND/OECX  are  essential,  because  it  is  able  to  help  local  authorities 
determine  the  specific  identity  of  the  UXO  and  whether  an  emergency  rendering  safe 
procedure  (RSP)  or  removal  of  UXO  is  required.  If  an  emergency  situation  exists, 
Huntsville  Division  will  assist  in  coordinating  the  activities  of  military  EOD  units  and 
securing  necessary  resources  to  protect  health  and  safety.  If  there  is  no  emergency,  OECX 
wiU  provide  initial  guidance  on  methods  to  secure  the  UXO  site,  prevent  the  development 
of  an  emergency  situation,  and  initiate  appropriate  next  steps.  Those  next  steps  depend 
upon  the  jurisdiction  in  which  the  UXO  has  been  discovered  but  generally  follow  the 
generic  outline  above.  The  major  differences  in  specific  steps  result  from  a  determination 
that  a  Hme-Critical  or  Non-Time-Critical  Removal  Action  is  required.  In  the  case  of  a 
Time-Critical  Removal  Action,  periods  for  public  comment  and  lengthy  public  participation 
in  the  decision  making  process  are  abbreviated. 


^  ^  Extracted  from  “Ordnance  and  Explosive  Waste  (OEW)  Mandatory  Center  of  Expertise  (MCX)  Interim 
Guidance  Regarding  Coordination  with  Explosive  Oidnance  Disposal  (EOD)  Organization, 
Enclosure  1,”  CEHND-PM-OMC  (385-16b),  dated  14  December  1994,  available;  http:/Av2.hnd.usace. 
armyjnil/oew/policy/coorencl  Jitml;  copy  on  file  at  IDA. 
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2.  Phase  1:  Preliminary  Assessment 

At  some  point  in  the  life  cycle  of  a  present  or  former  DoD  facility,  it  becomes 
appropriate  to  examine  continuing  need  for  the  facility.  At  that  time,  an  assessment  is  often 
begun  for  the  puipose  of  understanding  what  activities  may  have  taken  place  over  the  life  of 
the  facility  and  the  impact  of  such  activities  on  the  health,  safety,  and  environment  of  the 
surrounding  area.  If  the  facility  had  previously  been  associated  with  ordnance,  explosives, 
or  explosive  waste  as  the  result  of  RDT&E,  manufacturing,  or  stockpiling  activities,  there 
is  a  strong  possibility  that  environmental  remediation  and/or  restoration  activities  will  be 
required.  Such  requirements  might  arise  imder  the  IRP  (the  DoD  “Superfund”  program 
intended  to  bring  DoD  pollution  of  its  neighboring  territories  under  control)  or  from 
realignment  or  conversion  of  the  DoD  facilities. 

During  the  Preliminary  Assessment  Phase,  a  rigorous  search  of  all  known  records 
of  an  installation  is  undertaken.  *2  A  records  search  is  conducted  to  gain  as  much  informa¬ 
tion  as  possible  about  the  types,  quantities,  and  distribution  of  ordnance  that  can  be 
expected  on  the  site.  The  methods  used  and  the  reliability  of  these  record  searches  vary 
greatly  depending  on  the  circumstances.  Searches  can  take  only  a  few  days  or  up  to 
6  months.  Some  sites  have  complete  and  accurate  records  that  go  back  several  decades, 
whereas  others  have  more  haphazard  records. In  some  cases,  records  are  practically 
nonexistent  and  information  must  be  obtained  from  indirect  sources,  such  as  interviews 
with  personnel  that  once  worked  on  a  particular  site.  The  accuracy  of  the  records  also 
varies  a  great  deal. 


See  http://dogbert.ncr.usace.army.mil/ceteam/asr.htm  [Online]  for  an  excellent  overview  of  the  Archive 
Search  Report  process,  including  an  abbreviated  discussion  of  the  principles  and  benefits. 

See,  for  example,  Ordnance  and  Explosive  Waste  Archives  Search  Report  for  Assateague  Island, 
Worcester  County,  Maryland,  and  Accomack  County,  Virginia,  by  the  U.S.  Army  Corps  of  Engineers, 
Rock  Island  District,  and  the  U.S.  Army  Defense  Ammunition  Center  and  School,  dated  June  1994. 
See  also  Ordnance  and  Explosive  Waste  Archives  Search  Report  for  the  Former  Bluebonnet  Ordnance 
Plant,  McGregor,  Texas,  by  the  U.S.  Army  Corps  of  Engineers,  Rock  Island  District,  and  the  U.S. 
Army  Defense  Ammunition  Center  and  School,  dated  April  1994.  This  document  is  accessible  from 
the  U.S.  Army  Corps  of  Engineers  server  maintained  as  part  of  the  Corps  of  Engineer’s  efforts  to 
support  the  Defense  Environmental  Remediation  Program  ^REP). 
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In  the  case  of  active  ranges,  records  generally  have  been  kept  for  about  the  last 
20  years.  Ordnance  that  was  fired  before  that  time  is  often  not  recorded.  Current 
procedures  require  a  unit  to  supply  paperwork  describing  the  quantities  and  types  of 
ordnance  that  will  be  fired  on  each  range  prior  to  use.  However,  this  was  not  always  the 
case.  Furthermore,  the  quality  of  records  varies  widely,  even  among  ranges  with  similar 
missions.  Army  Corps  of  Engineer  records  of  Ordnance  and  Explosive  remediation 
efforts*  5  and  EOD  persormel  identified  the  following  general  trends  in  record  keeping: 

•  The  least  busy  ranges  generally  have  the  best  records. 

•  Chemical  weapons  testing  has  always  been  well  documented,  even  for 
ordnance  that  was  fired  before  current  requirements  mandating  ordnance 
records  were  in  place.*® 

•  Records  are  usually  more  accurate  for  routine  maintenance  cleanup,  since  a 
Military  Range  or  base  generally  has  accurate  information  about  what  has  been 
fired  in  the  approximately  1-year  time  span  between  subsequent  sweeps. 
Records  from  more  than  about  20  years  ago  can  be  sparse,  inaccurate,  or  both. 

•  Test  and  evaluation  ranges  tend  to  have  fairly  complete  records.  Records  are 
generally  very  good  for  developmental  tests  of  small  numbers  of  items.  For 
example,  EOD  personnel  identified  White  Sands  as  an  example  of  a  base 
having  records  that  are  detailed  and  complete.  Such  records  generally  give  the 
EOD  personnel  the  information  they  require  in  a  short  time  frame,  usually  only 
a  few  days. 

•  Record  keeping  associated  with  historic  ordnance  waste  disposal  on  base  is 
frequently  incomplete,  inaccurate,  or  nonexistent.  In  years  past,  ammunition 


*^  See,  for  example,  the  discussion  of  the  1991  Preliminary  Assessment  of  Assateague  Island  in  U.S. 
Army  Corps  of  Enginem  Rock  Island  District,  and  U.S.  Army  Defaise  Ammunition  Center 
and  School,  Ordnance  and  Explosive  Waste  Archives  Search  Report  for  Assateague  Island,  Worcester 
County,  Maryland,  and  Accomack  County,  Virginia  (Project  Number  C03MD093001,  June  1994), 
Section  2,  available:  ht^://dogbeit.ncr.usace.army.mil/military/deip/fuds/projects/ASSATEAG/oew 
/asr/  findings/dp09toc.btm#secl ;  copy  on  file  at  IDA. 

*^  See,  for  example,  records  of  selected  Army  Corps  of  Engineers  ordnance  and  explosive  waste  remedi¬ 
ation  archive  record  searches  for  the  following  facilities:  Assateague  Island,  Maryland,  and  Virginia 
(cited  above);  Bluebonnet  Ordnance  Plant,  McGregor,  Texas,  available:  http://dogbett.ncr.usace.army. 
mil/tnilitary/detp/fuds/ptojects/bluebone/bluebone.htm);  Camp  Croft,  Spartanburg,  South  Carolina, 
(available:  htlp://dogbert.ncr.usace.army.mil/military/detp/fu(Wprojects/cmpcroft/cmpctoft.htm);  Camp 
(jteene,  Charlotte,  North  Carolina  (available:  http://dogbert.ncr.usace.aimy jnil/military/derp/fud^ 

projects/cmpgreen/cmpgreen  Jitm);  and  Culebra  Island  National  Wildlife  Refuge,  Culebra,  Puerto  Rico 
(available:  bttp://dogbett.ncr.usace.armyjiiil/military/detp/fuds/projects/culebra/culebraJitm). 

*®  See,  for  example.  Volume  I,  Archives  Search  Report  and  Appendices  A-F,  Supplmental  Ardiives 
Search  Report,  Preliminary  Assessment  of  Chemic^  Warfare  materials  at  the  Former  Black  Hills  Army 
Depot,  South  Dakota  (available:  http://dogbert.ncr.usace.army.tnil/tnilitary/derp/fuds/projects/black_hi/ 
black_hi.htm). 
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was  often  disposed  in  burial  pits,  the  locations  of  which  were  not  accurately 
recorded  or  records  of  which  were  not  preserved.  Current  regulations  prohibit 
these  practices,  but  locating  and  cleaning  up  the  residue  from  the  past  is  a 
challenge  DoD  must  meet 

•  Sometimes  ordnance  inadvertently  landed  outside  the  desired  impact  area 
without  being  observed  or,  if  observed,  without  having  been  recorded. 

The  EOD  personnel  with  whom  we  spoke  stressed  that  immaculate  records  do  not 
necessarily  correspond  to  conditions  found  in  the  field.  There  can  be  uncertainties  in  what, 
where,  and  how  much  ordnance  is  found,  even  on  ranges  with  “good”  records.  Other 
problems  with  documentation  may  arise  under  special  circumstances.  For  example,  aUied 
troops  conducting  training  on  American  ranges  with  foreign-made  ammunition  may  not 
record  their  activities  with  the  same  degree  of  rigor  and  accuracy  as  U.S.  forces.  Testing 
of  special  ordnance  that  is  not  yet  in  the  inventory  and  firing  foreign  ordnance  for 
technology  exploitation  may  also  result  in  incomplete  range  use  documentation.  Intelli¬ 
gence  and  special  operations  activities  may  leave  incomplete  records  of  range  use,  consis¬ 
tent  with  the  requirements  of  operations  security  associated  with  their  activities.  These 
situations  can  often  leave  UXO-qualified  personnel  without  complete  information  about 
what  to  expect  when  entering  a  range  or  other  DoD  facility  for  the  purpose  of  clearing 
UXO. 

The  development  of  new  information  processing  technology  and  techniques  holds 
out  the  promise  of  making  record  searches  in  support  of  UXO  clearance  and  other 
environmental  remediation  efforts  much  more  efficient  and  effective.  The  Corps  of 
Engineers  is  integrating  a  Geographic  Information  System  (GIS)  image  processing  system 
and  database  as  part  of  its  Ordnance  and  Explosive  Waste  (OEW)  clearance  program.  The 
information  system  is  part  of  the  Corps  FUDS  program.  This  approach  integrates  informa¬ 
tion  from  a  variety  of  historical  sources  and  records  searches.  For  example,  maps  from  the 
National  Archives  and  the  U.S.  Geological  Survey  (USGS)  can  be  used  to  catalog  past 
uses  of  a  EtoD  facility.  Present  maps,  showing  current  structures  and  geographical 
features,  can  be  overlaid  on  these  historical  maps.  Engineers  can  compare  and  contrast 
current  structures  and  activities  with  past  uses  of  the  installation.  This  reduces  the  area  to 
be  subjected  to  a  detailed  site  survey  by  focusing  efforts  on  areas  with  a  high  probability  of 
ordnance  contamination.  Projects  at  Fort  Monroe  and  Camp  Simms  have  demonstrated  the 
use  of  this  approach  with  good  results. 

The  use  of  historical  records  in  defining  the  characteristics  of  a  site  on  which  UXO 
contamination  may  be  present  is  important  but  should  not  be  overstated.  There  seems  to  be 
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a  perception  in  the  sensor  development  community  that  copious  quantities  of  accurate, 
reliable,  and  complete  historical  information  for  each  site  of  possible  UXO  contamination 
are  available.  Sensor  developers  often  suggest  that  their  products  will  work  well  when 
tuned  to  a  particular  ordnance  type,  implying  that  the  UXOqualified  personnel  engaged  in 
clearing  operations  will  know  with  precision  which  type  of  ordnance  to  expect  or,  alter¬ 
natively,  that  only  ordnance  of  a  particular  type  will  be  encountered  when  clearing  a 
specific  range.  We  spoke  with  the  EOD  personnel  who  had  experience  in  range  clearing 
and  UXO  clearance  at  FUDS  locations.  They  confirmed  that  record  searches  were  usually 
not  entirely  accurate  in  characterizing  the  UXO  actually  found  during  clearing  operations. 

3.  Phase  2:  Remedial  Investigation/Engineering  Evaluation 

After  the  completion  of  a  Preliminary  Assessment  of  a  site,  including  a  detailed 
historical  survey,  a  detailed  assessment  is  undertaken.  The  Site  Survey  phase  is  conducted 
by  UXO-qualified  and  support  personnel  for  the  purpose  of  trying  to  locate  UXO, 
characterize  its  condition,  and  gain  additional,  site-specific  information  necessary  to  draw 
up  detailed  engineering  alternatives.  Detailed  site  surveys  are  intended  to  fulfill  the 
following  purposes: 

•  Identify  most  likely  areas  of  anomalies  indicating  UXO  or  other  contamination. 

•  Develop  specific  requirements  for  equipment,  materiel,  and  specialized 
additional  support  personnel. 

•  Detect  the  presence  of  additional  hazardous  materials,  structures,  utilities, 
antiquities,  or  the  remains  of  other  human  activity  that  may  not  have  been 
adequately  described  in  the  archive  search  reports. 

•  Develop  a  detailed  health  and  safety  plan,  including  emergency  medical  and 
materiel  response  plans  in  the  event  of  accident  or  contamination. 

During  the  early  field  investigation  phase,  several  100  meter  by  100  meter  plots  will 
be  marked  and  searched  to  develop  an  estimate  of  the  total  contamination  on  the  site. 
UXO-related  procedures,  including  the  location  and  marking  of  subsurface  anomalies  and 
the  location  and  maiking  of  suspected  surface  UXO,  will  be  performed  during  investigation 
of  these  sample  plots.  These  activities  may  be  performed  by  either  UXO-qualified 
personnel  or  others  who  are  not  UXO  qualified.  Certain  UXO  procedures  may  also  be 
undertaken,  including  (1)  gaining  access  to  (manual  excavation)  and  identifying  subsurface 
anomalies  and  assessing  condition  of  buried  UXO  and  (2)  identifying  and  assessing  the 
condition  of  surface  UXO.  Only  UXO-qualified  personnel  (who  may  be  uniformed 
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military  personnel  or  civilian  munitions  destroyers)  are  allowed  to  perform  these  UXO 
procedures.*"^ 

The  Site  Survey  results  in  the  collection  of  detailed  information  which  will  confirm 
the  findings  of  the  Archive  Records  Search  and  Preliminary  Assessment,  expand  upon 
them  (in  terms  of  munitions  types,  condition  of  UXO,  etc.),  and  provide  additional  data 
needed  to  accurately  gauge  the  most  effective  routes  to  removal  of  UXO  and  other 
contaminants  on  the  site.  These  alternatives  are  developed  in  subsequent  phases  of  the 
environmental/installation  remediation/restoration  action. 

4.  Phase  3:  Feasibility  Study/Cost  Analyses 

A  second  round  of  studies,  analyses,  and  site  surveys  may  be  required  to  arrive  at  a 
set  of  feasible  environment  restoration/remediation  alternatives  which  can  be  examined  for 
cost  and  schedule  implications.  Substantial  additional  site  surveys,  investigations  into  the 
actual  condition  of  UXO,  and  location  and  marking  of  subsurface  anomalies  may  be 
required.  This  phase  of  UXO  clearing  is  almost  always  undertaken  in  environmental 
restoration  activities  associated  with  either  IRP  or  BRAC  actions.  It  may  also  be  included 
in  FUDS  cleanup  where  emergency  actions  are  not  required. 

5.  Phase  4:  Removal  Design  (Explosive  Safety  Submission) 

Following  completion  of  the  initial  phases  of  UXO  clearance,  and  the  selection  of 
an  approach  to  the  elimination  of  UXO  contamination,  a  detailed  Removal  Design  and 
Safety  Plan  is  prepared  and  submitted  to  the  Headquarters,  Corps  of  Engineers,  and  the 
DoD  Explosives  Safety  Board.  Preparation  of  the  Removal  Design  and  Safety  Plan  may 
require  additional  site  visits  and  extensive  exploration  to  further  identify  types  of  UXO  and 
constituents  present,  their  overall  condition,  other  hazards  or  potential  hazards  commingled 
with  the  UXO,  and  the  response  capability  of  the  facility  and  its  surrounding  environment 
in  the  event  of  an  accident  or  environmental  hazard  release  during  UXO  contamination 
removal.** 

Embedded  within  the  Removal  Design  and  Safety  Plan  is  a  detailed  Health  and 
Safety  Plan.  This  document  outlines  the  specific  proposed  measures  that  will  be  carried  out 


*"*  See  Reference  9,  paragraph  3.  Definitions. 

**  The  specific  standards  for  the  health  and  safety  plan  are  set  forth  in  Occupational  Health  and  Safety 
Administration  (OSHA)  standards  published  at  29  CFR  1910.120  and  1926;  and  U.S.  Army  Corps  of 
Engineers,  EM  385-1-1,  Safety  and  Health  Requirements  Manual. 
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during  the  day-to-day  process  of  UXO  clearance  operations.  The  broad  outline  of  the 
Health  and  Safety  Plan  is  the  following:!^ 

1.  Site  Description  and  Contamination  Characterization. 

2.  Hazard/Risk  Analysis. 

3.  Accident  Prevention. 

4.  Staff  Organization,  Qualifications,  and  Responsibilities. 

5.  Training. 

6.  Personal  Protective  Equipment 

7.  Medical  Surveillance. 

8.  Exposure  Monitoring/Air  Sampling  Program. 

9.  Heat/Cold  Stress  Monitoring. 

10.  Standard  Operating  Safety  Procedures;  Engineering  Controls  and  Work 
Practices  as  Appropriate. 

1 1 .  Site  Control  Measures. 

12.  Personal  Hygiene  and  Decontamination. 

13.  Equipment  E)econtamination. 

14.  Emergency  Equipment  and  First  Aid  Requirements. 

15.  Emergency  Response  and  Contingency  Procedures  (On  Site  and  Off  Site). 

16.  Logs,  Reports,  and  Record  Keeping. 

Much  of  the  work  undertaken  to  perform  the  Preliminary  Assessment  and  to 
prepare  other  Engineering  Evaluation,  Feasibility,  and  Cost  Estimate  documents  would  be 
directly  applicable  to  the  preparation  of  the  Removal  Action  and  Safety  Plan  and  the 
subordinate  Health  and  Safety  Plan.  However,  further  UXO  procedures  may  be  required 
on  the  initial  sample  area  to  acquire  all  information  needed  to  prepare  the  safety  and  health 
plan  for  decontamination  of  the  site. 


This  is  based  on  “i^pendix  B,  Safety  and  Health  Elements  for  HTRW  Documents,”  in  U.S.  Anny 
Corps  of  Engineers,  Engineer  Regulation  385-1-92,  Safety  and  Occupational  Health  Document 
Requirements  for  Hazardous,  Toxic,  and  Radioactive  Waste  (HTRW)  Activities  (Washington,  D.C., 
13  December  1991). 
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6.  Phase  5:  Removal  Action 

Months  after  the  initial  discovery  of  UXO,  a  detailed  plan  for  the  removal  of  UXO 
contamination  at  a  site  has  been  prepared  and  reviewed  by  government  and  citizen  groups, 
and  resources  have  been  marshaled  to  restore  the  environment.  The  removal  actions  now 
proceed  with  full-blown  UXO  reconnaissance — detailed  survey  of  the  entire  site. 

During  the  course  of  conducting  a  detailed  site  survey,  special  care  must  be 
exercised  to  avoid  contact  with  camouflets,  small  underground  cavities  formed  when 
ordnance  penetrates  the  earth  to  a  depth  where  the  force  of  the  explosion  is  insufficient  to 
rupture  the  earth’s  surface.  Camouflets  are  filled  with  carbon  monoxide  gas  and  other  end 
products  of  ordnance  explosion  and  can  be  very  dangerous  to  personnel  seeking  to  clear 
UXO. 

Access  to  the  detected  UXO  is  gained  by  UXO-qualified  personnel  by  excavation 
using  power  equipment,  powered  hand  tools,  or  by  hand  in  accordance  with  the  Removal 
Action  and  Safety  Plan.  The  identity  of  the  munition  is  verified.  The  type  and  condition  of 
the  fuzing  and  arming  mechanisms,  the  condition  of  explosive  materials  in  the  munition, 
and  the  condition  of  other  materials  within  the  munitions  case  are  evaluated. 

The  UXO-qualified  specialist  then  examines  the  surrounding  environment  to 
determine  whether  the  safety  of  the  UXO  clearance  team  and  other  personnel  would  be 
endangered  from  an  unsuccessful  in-place  Render  Safe  Procedure  (RSP).  Even  though 
these  issues  have  been  addressed  in  the  extensively  documented  Removal  Action  and 
Safety  Plan,  variations  from  the  expected  conditions  based  on  preliminary  assessments  and 
site  surveys  to  the  specific  conditions  on  hand  require  constant  attention  to  safety. 

The  preferred  method  of  UXO  clearance  is  disposal  in  place  by  detonation.  If  this 
cannot  be  accomplished,  open  burning  of  explosives  and  smokeless  powder,  or  chemical 
decomposition  of  explosives  may  be  allowed,  but  prior  approval  of  the  contracting  officer 
is  required.  If  necessary,  UXO  may  be  transported  off  site  for  proper  disposal.  If 
transportation  is  required,  UXO  must  be  treated  as  hazardous  cargo  in  accordance  with 
49  CFR  100-199,  DA  Pam  385-64,  and  relevant  state  and  local  laws. 

F.  UXO  CLEARANCE:  SUMMARY 

This  chapter  summarizes  the  general  processes  by  which  UXO  contamination 
comes  to  light  and  is  dealt  with  within  CONUS  by  DoD  Components. 
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Within  CONUS,  the  primary  focus  of  UXO  cleanup  has  been  on  ferrous  and  non- 
ferrous  metal  munitions  and  chemical  components.  A  1995  Congressional  Budget  Office 
study  observed  the  following: 

Cleaning  up  unexploded  ordnance  and  chemical  warfare  materials  is  among 
the  most  difficult,  dangerous,  time-consuming,  and  expensive  tasks  DoD 
faces.  The  U.S.  Army  Corps  of  Engineers  has  identified  almost  1,700  sites 
on  which  these  hazardous  materials  have  been  reported. 

Current  technology  to  remediate  buried  ordnance  is  time  consuming  and 
costly.  Most  ordnance  sites  are  surveyed  by  operators  on  foot  using  hand¬ 
held  metal-detecting  equipment.  Bulldozers  and  specially  protected  heavy 
equipment  are  used  to  dig  up  buried  ordnance  and  transport  it  to  facilities 
where  it  will  be  de-armed  or  exploded....  DoD  recently  estimated  that, 
using  current  technology,  it  costs  about  $65,000  per  acre  to  survey  and 
remediate  a  site  with  unexploded  buried  ordnance.  The  Army  Crops  of 
Engineers  estimates  that  tens  of  thousands  of  acres  will  require  remediation. 
Cleanup  costs  for  buried  ordnance  and  chemical  warfare  materials  could 
total  several  billion  dollars.^® 

The  key  to  accelerating  the  pace  of  CONUS  cleanup  and  reducing  its  overall  cost  to 
DoD,  while  accelerating  the  transfer  of  surplus  DoD  facilities  to  others,  may  be  found  in 
developing  new  technology  which  is  better  able  to  detect,  classify,  and  identify  buried 
objects  as  UXO  requiring  remediation.  We  turn  to  a  consideration  of  the  technology  needs 
of  the  UXO  remediation  community. 


See  Congressional  Budget  Office,  Ckaning  Up  Defense  Installations:  Issues  and  Options, 
(Washington,  D.C.,  CBO,  January  1995),  downloaded  from  the  Internet  but  no  longer  available,  copy 
on  file  at  IDA. 
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III.  NEEDS  OF  THE  UXO  REMOVAL  COMMUNITY 


In  this  chapter  we  report  on  a  set  of  emerging  technical  needs  that  successful  UXO 
detection  technology  should  be  able  to  meet 

It  is  important  to  bear  in  mind  that  the  ESTCP  is  a  DoD  demonstration  and  valida¬ 
tion  program.  It  is  therefore  obliged  to  consider  operational,  maintenance,  and  training 
requirements  or  implications  of  technology  included  in  its  demonstration  and  validation 
activities.  While  the  potential  concepts  of  operation,  training,  maintainability,  and 
reliability  issues  need  not  be  completely  resolved  during  field  demonstrations,  the  impact  of 
these  considerations  on  users  has  a  strong  bearing  on  their  receptiveness  to  participate  in  a 
demonstration/validation  (dem/val)  program  and  to  lobby  on  behalf  of  an  emerging  tech¬ 
nology  that  represents  a  technical  advance.  Those  technologies  which  demonstrate  promise 
in  narrow  technical  parameters  but  do  not  relieve  operational,  logistical,  or  training  burdens 
on  operators  do  not,  as  a  general  rule,  survive  in  the  acquisition  process 

IDA  staff  met  several  times  with  DoD  EOD  Program  Managers  and  trainers  because 
DoD-trained  EOD  personnel  constitute  virtually  the  entire  current  and  near-term  pool  of 
UXO-qualified  personnel.  We  wished  to  better  understand  the  operational  needs  for  and 
impediments  to  the  adoption  of  new  UXO  detection  technology  for  peacetime,  CONUS- 
based  UXO  clearance  from  the  perspective  of  the  EOD  community.  Our  goal  was  to  assess 
the  ESTCP  UXO  Technology  Demonstration  Plan  and  offer  suggestions  and  recommenda¬ 
tions  that  took  concerns  of  field-level  operatives  into  consideration.  We  did  not  address 
substantive  resource  management  questions  in  detail  because  those  larger  policy  and 
personnel  issues  are  well  beyond  the  scope  of  this  task.  Nevertheless,  in  focusing  on  the 
development  of  technology  and  testing  methods  to  use  existing  resources  more  efficiently, 
we  learned  of  policy  and  technology  concerns  of  users  that  will  have  an  impact  on  the 
ultimate  acceptance  of  and  insertion  of  new  UXO  detection  technology  into  DoD  activities. 
The  following  discussion  summarizes  these  concerns  as  well  as  the  technical  performance 
needs  of  the  UXO  community. 

A.  POLICY  AND  HUMAN  RESOURCES 

The  ability  to  clear  UXO  from  DoD  facilities  and  FUDS  within  CONUS  is  limited, 
at  least  in  part,  by  the  availability  of  UXO-qualified  personnel.  Current  DoD  policies  allow 
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non-UXOqualified  personnel  to  perform  the  following  functions  associated  with  removal 
of  UXO  contamination  from  active  DoD  facilities  or  FUDS: 

•  Location  and  marking  of  subsurface  anomalies. 

•  Location  and  marking  of  suspected  surface  UXO. 

•  Transportation  and  storage  of  recovered  UXO. 

•  Utilizing  earth-moving  machinery  (EMM)  to  excavate  soil  to  no  closer  than 
approximately  12  inches  of  a  subsurface  anomaly. 

UXO-qualified  personnel  must  carry  out  the  following  functions: 

•  Gaining  access  to  (manual  excavation)  and  identifying  subsurface  anomalies, 
and  assessing  condition  of  buried  UXO. 

•  Identifying  and  assessing  condition  of  surface  UXO. 

•  Recovery  and  final  disposal  of  all  UXO.' 

Reallocation  of  responsibilities  from  UXO-qualified  to  non-UXO  qualified  person¬ 
nel  or  expanding  the  number  of  UXO-qualified  personnel  through  reliance  on  civilian 
training  programs  might  have  some  positive  effect  on  the  shortage  of  personnel  trained  in 
UXO  clearance.  To  the  extent  that  UXO  clearance  is  limited  only  by  the  availability  of 
UXO-qualified  personnel,  these  policy  options  might  be  worth  assessing  in  detail. 

However,  discussions  with  UXO  technology  users  led  us  to  conclude  that  the 
development  of  new  UXO  detection  technology  will  not  lead  to  insertion  and  acceptance  of 
by  the  user  community  unless  it  offers  operational  advantages.  Users  told  us  that  improve¬ 
ments  in  safety,  reliability,  ease  of  maintenance,  and  ease  of  use  would  be  important  factors 
in  their  decisions  to  accept  new  technology  in  fielded  UXO  detection  systems.  The  impact 
of  new  technology  on  training  requirements,  maintenance  burden,  and  non-time-critical 
response  might  become  a  salient  issue  and  a  barrier  to  the  acceptance  of  demonstrated  UXO 
detection  technology  in  the  future. 

Based  on  our  review  of  existing  literature,  as  well  as  discussions  with  EOD 
personnel  at  DoD  ranges,  we  believe  improvements  in  UXO  clearance  efficiency  require 
improvements  in  two  technology  areas  which  translate  immediately  into  improved  safety 
for  personnel  involved  in  UXO  clearing  operations.  Increases  in  detection  capability 
accompanied  by  improvements  in  the  ability  to  discriminate  hazardous  ordnance  from  other 
debris  will  be  well  received  by  users.  Further,  minimization  of  false  alarms  of  all  types 


'  See  U.S.  Army  Corps  of  Engineers,  Huntsville  Division,  “Interim  Guidance  Safety  Concepts  and  Basic 
Considerations  for  Unexploded  Ordnance  (UXO)  Operations,”  Revised  16  February  1996 
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will  also  be  perceived  as  an  operational  advantage  compared  to  present  technology. 
Accomplishing  these  technical  objectives  will  improve  the  overall  efficiency  of  UXO 
clearance. 

B.  SENSORS 

The  primary  sensor  used  by  EOD  technicians  to  detect  UXO  today  is  the  magne¬ 
tometer.  Active  sensors  such  as  induction  coils  and  ground-penetrating  radar  (GPR)  are 
also  used  on  occasion.  EOD  personnel  expressed  concerns  that  the  signals  emitted  from 
active  devices  could  trigger  electronic  fuzes,  especially  whenever  fuzes  have  been 
damaged.  There  appears  to  be  poor  understanding  of  the  possible  interactions  between 
UXO  detector  sensors  and  fuzes  among  the  EOD  and  the  UXO  detection  communities. 
The  possibility  of  deliberate  or  accidental  initiation  of  arming  and  fuzing  actions  in  UXO  as 
a  consequence  of  active  sensor  UXO  should  be  addressed  during  the  development  of  any 
active  sensor  system. 

The  magnetometers  most  commonly  used  in  UXO  work  today  typically  give  the 
technician  many  targets  to  evaluate,  but  the  vast  majority  of  these  are  from  debris  rather 
than  ordnance.  Personnel  involved  in  the  detection,  marking,  and  identification  of  UXO 
need  a  sensor  that  can  distinguish  the  actual  hazards  from  debris,  so  targets  can  be  evalu¬ 
ated  and  appropriate  courses  of  action  chosen.  A  sensor  that  could  provide  information 
about  the  shape  of  the  object,  whether  it  was  intact,  whether  it  contained  high  explosive, 
and  whether  the  fuzing  was  active  would  be  nearly  ideal. 

The  development  and  fielding  of  such  an  ideal  sensor  may  be  years  away.  In  the 
near  term,  incremental  improvements  in  performance  for  detection  and  discrimination  of 
targets  and  debris  would  be  valuable.  The  Jefferson  Proving  Ground  demonstration 
established  a  baseline  performance  level  for  a  variety  of  currently  available  and  emerging 
technologies.^  The  most  successful  demonstrators  achieved  probabilities  of  detection  {Pd) 


^  PRC,  Inc.,  Demonstrator  Work  Plan  for  UXO  Detection,  Identification  and  Remediation  Advanced 
Technology  Demonstration  at  the  Jefferson  Proving  Ground,  Indiana  (NAVEODTECHCEN  Contract 
No.  N00600-88-D-3717,  1994);  U.S.  Army  Environmental  Center,  Unexploded  Ordnance  Advanced 
Technobgy  Demonstration  at  Jefferson  Proving  Ground  (Phase  I)  (piqiaredby  PRC,  Inc.,  Report  No. 
SFIM-AEC-ET-CR-94120,  December  1994);  M.  Mulqueen,  V.  George,  A.  Andrews,  D.  Sparrow, 
and  R.  Dugan,  “Performance  Assessment  at  the  Jefferson  Proving  Ground  Demonstration  of  Systems 
for  the  Detection  and  Identification  of  Buried  Unexploded  Ordnance,”  SPIE  Proceedings,  April  1995; 
U.S.  Army  Enviroiunental  Center,  Evaluation  of  Individual  Demonstrator  Performance  at  the 
Unexploded  Ordnance  Advanced  Technology  Demonstration  Program  at  Jefferson  Proving  Ground 
(prepared  by  the  Institute  fw  Defense  Analyses,  Report  No.  SFIOM-AEC-ET-CR-95033,  Match  1995); 
Institute  for  Defense  Analyses,  Demonstrator  Performance  at  the  Unexploded  Ordnance  Advanc^ 
Technology  Demonstration  at  Jefferson  Proving  Ground  (Phase  I)  and  Implications  for  UXO  Clearance, 
IDA  Paper  P-3114, 1995. 
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of  aboul  70  percent  and  one  or  mote  false  alarms  per  ordnance  item  detecl^. 
technicians  with  whom  we  spoke  indicated  that  this  capability,  althoush  far  from  ideal, 
would  be  useful  to  them.  For  example,  they  specifically  indicated  that  a  map  of  the  range 
with  a  known  and  reasonably  high  Pd  (70  percent)  would  direct  them  to  specific  locauons 
of  anomalies  to  be  investigated — a  tool  they  do  not  now  have. 

In  general  the  ECD  technicians  with  whom  we  spoke  seek  a  sensor  that  will  let 
tt«m  do  their  jobs  better,  faster,  or  more  safely  than  at  present.  They  do  not  necessarily 
want  a  silver  buUet  that  claims  to  solve  all  of  their  problems  at  once.  Rather,  they  would 
find  value  in  sensors  directed  at  specific  problems  that  can  do  one  job  well.  For  exampfe  a 
sensor  that  could  detect  UXO  in  very  rough  or  heavily  vegeuted  terrain,  where 
reconnaissance  is  very  difficult,  even  on  fool,  would  be  very  helpful.  For  such  a  develop¬ 
mental  sensor  to  be  of  value,  however,  substantial  documentauon  descnbing  the  effect  of 
weather  depth  of  UXO,  size  of  UXO,  ambient  environmental  condiuons,  etc.,  on  sensor 
performance  would  also  have  to  be  provided.  n.e  total  data  package  would  ensure  that 
technicians  neither  overestimate  nor  underestimate  the  effectiveness  of  the  sensor  when 
examining  data  collected  from  it 

Of  the  three  goals-“better,  faster,  more  safely”-EOD  personnel  consider  doing 
their  jobs  more  safely  to  be  the  most  important  The  EOD  personnel  with  whom  we  spoke 
indicated  that  the  majority  of  accidents  are  caused  by  human  error  dunng  the  UXO  remov 
phase  of  response  actions.3  Unfortunately,  our  interviewees  found  no  common  auts 
which  could  account  for  accidents  in  their  personal  experiences.  There  was  no  single  piece 
of  information  that  could  be  identified  by  our  interviewees  that  would  have  averted 

accidents  had  it  been  provided  by  a  sensor. 

New  technology  incorporated  into  new  tools  that  can  reduce  the  likeUhood  of 
accidents  is  most  likely  to  be  adopted  by  the  EOD  community.  Since  most  accidents  occur 
after  UXO  has  been  detected,  improvements  in  remote  identification  of  UXO  and  its  m  sttu 
characterization  are  improvements  most  likely  to  yield  benefits  in  terms  of  pemonnel  safety. 
Knowing  the  specific  munition,  the  condition  of  the  arming  and  fuzing  mechanisms,  the 
condition  of  the  explosive  material  contained  within  the  munition  and  its  immediate 
environs,  the  nature  of  the  material  adjacent  to  the  munition,  and  the  material  m  the 
surrounding  area  would  allow  UXO-qualifted  personnel  opportunities  to  muumtze 


ti  dionld  be  suessed  that  this  infoimation  was  taken  trout  a  relativety  small  number  ot  EOD  personnel, 
l! "  siWe  Sridenis  share  more  in  common  than  those  ot  this  gronp’s  expenence. 
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excavation  of  UXO  before  having  specific  render  safe/disposal/removal  plans  for  each 
UXO  target  in  hand. 

C.  COMMUNITY  ACCEPTANCE 

The  most  important  question  in  designing  a  test  of  this  equipment  is,  “What 
information  does  the  user  need  to  establish  the  utility  of  this  technologyT’  When  we  asked 
this  question  of  EOD  technicians,  there  was  substantial  agreement  that  they  would  first  like 
to  see  a  controlled  test  done  on  a  live  range.  Such  a  test  would  have  some  ordnance 
emplaced  with  known  ground  truth  so  that  there  is  a  measure  of  detection  capability.  This 
test  should  be  followed  by  a  demonstration  also  on  a  live  range  in  circumstances  similar  to 
those  under  which  the  technology  would  ultimately  be  used. 

The  EOD  community  would  like  to  test  the  equipment  in  a  test  and  evaluation 
(T&E)  unit  made  up  of  EOD  personnel.  Some  of  the  more  important  questions  that  should 
be  addressed  are  the  following: 

•  What  is  required  to  establish  and  maintain  user  competency? 

•  What  kind  of  ancillary  support  (power,  cooling,  transportation)  is  required  to 
sustain  the  equipment? 

•  What  circumstances  are  stressing  to  the  equipment? 

Finally,  EOD  personnel  are  interested  in  an  assessment  similar  to  a  Cost  and  Operational 
Effectiveness  Analysis  (COEA)  to  establish  time  between  failure,  cost,  servicing  require¬ 
ments,  and  so  on. 

The  EOD  community  emphasized  the  need  for  controUed  data  collection  and 
analysis  with  independent  verification  and  validation  of  claims  regarding  successful  detec¬ 
tion  of  UXO  and  discrimination  of  UXO  from  other  debris.  We  came  away  from  inter¬ 
views  with  the  distinct  impression  that  the  EOD  community  felt  victimized  by  developer 
claims  which  turned  out  to  be  false. 

There  was  also  some  discussion  regarding  the  desirability  of  operating  UXO 
detection  systems  in  a  manner  consistent  with  current  practices  and  sound  safety  practices. 
Much  UXO  activity  undertaken  today  by  EOD  personnel  is  regarded  as  training  for  EOD 
wartime  missions.  To  the  extent  that  such  training  compels  development  of  man-portable 
UXO  detection  systems,  the  ESTCP  technology  demonstration  does  not  lend  itself  to  the 
development  of  new  combat  support  equipment  However,  since  UXO  clearance  for  IRP, 
BRAC  facilities,  and  FUDS  does  not  generally  relate  directly  to  wartime  EOD  missions,  the 
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ESTCP  technology  demonstration  may  have  only  limited  immediate  relevance  to  formal 
EOD  technology  requirements  as  promulgated  through  the  JCS/CINC  process. 

On  the  other  hand,  high  costs  and  pressing  resource  demands  for  efficient,  reliable 
UXO  detection  in  support  of  range  clearance,  installation  rehabilitation,  closure  and  n^sfer 
of  installations  as  a  result  of  the  BRAC  process,  and  clearance  of  UXO  at  FUDS  are 
compelling  arguments  for  continued  DoD  investment  in  UXO  detection  technology  demon¬ 
strations.  Furthermore,  if  a  technology  can  meet  the  information  needs  for  UXO  detection, 
discrimination,  identification,  and  characterization  in  the  IRP/BRAC/FUDS  environment,  it 
may  pave  the  way  for  the  development  of  similar  technology  packaged  to  meet  the  needs  of 
combat  EOD  units.  Hence,  the  ESTCP  technology  demonstration  remains  an  important 
facet  of  EtoD’s  ongoing  RDT&E  efforts  to  meet  current  and  future  combat  technology 
needs. 
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IV.  TEST  DESIGN 


This  chapter  incorporates  the  lessons  learned  at  previous  UXO  detection  technology 
demonstrations,  with  the  types  of  information  that  UXOqualified  personnel  find  useful  in 
assessing  a  new  technology.  It  draws  heavily  upon  the  UXO  detection  experiments 
conducted  at  the  Jefferson  Proving  Ground,  Indiana.  This  section  summarized  the  user 
needs  that  might  be  addressed  with  new  technology  or  new  systems  and  outlines  the  design 
of  technical  tests  to  collect  data  addressing  the  MOEs  of  interest  to  EDD  personnel.  We 
will  sketch  how  the  issue  of  training  load  vs.  system  effectiveness  can  be  addressed  in 
these  tests.  In  addition,  there  are  known  pitfalls  in  any  demonstration  of  this  type.  We  will 
review  some  of  the  dangers  and  their  remedies.  Finally,  no  real-world  test  with  con¬ 
strained  resources  wiU  suffice  to  answer  all  the  questions  one  might  ask.  Hence,  we  will 
close  with  recommendations  for  questions  that  will  probably  need  a  parametric  study  or 
COEA-type  study. 

A.  QUESTIONS  TO  BE  ADDRESSED 

The  two  most  important  measures  of  performance  for  a  detection  system  are  the 
probability  of  detection  of  an  object  and  the  probability  of  detecting  a  “real”  object  versus 
the  probability  of  detecting  a  “false”  target  When  a  detector  reports  a  target  it  is  not 
immediately  obvious  whether  the  report  is  a  hit  (a  real  target)  or  a  false  alarm.  For 
example,  how  near  must  the  hit  be  to  a  target  object  to  count  as  a  detection? 

There  is  also  a  question  as  to  the  definition  of  a  false  alarm.  Some  believe  that 
detecting  a  man-made  nonordnance  object  that  falls  within  the  sensor  detection  parameters 
should  be  counted  as  a  hit  even  though  exploitation  of  the  data  would  result  in  the 
excavation  of  a  nonordnance,  nonhazardous  item  (and  the  consumption  of  scarce  human 
and  material  resources).  Just  because  a  sensor  works  within  design  parameters  and  reports 
a  buried  object  does  not  mean  that  it  has  detected  UXO.  Accordingly,  we  recommend  that 
all  nonordnance  detections  be  counted  as  false  alarms. 

From  an  operational  point  of  view,  all  resources  spent  recovering  nonhazardous 
items  are  wasted.  It  is  therefore  useful  to  distinguish  man-made,  nonordnance  items  from 
“natural”  false  alarms.  This  information  will  be  important  if  results  are  to  be  extrapolated  to 
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predict  performance  at  other  locations.  The  extrapolation  will  be  greaUy  aided  by  back¬ 
ground  data  taken  on  uncontaminated  sites.  The  ability  to  distinguish  man-made,  non¬ 
ordnance  items  from  naturally  occurring  ferrous  materials  which  generate  false  alarms  will 
also  be  helpful  in  planning  further  environmental  remediation/restoration  efforts  as  may  be 
required,  depending  upon  the  future  uses  of  the  sites  from  which  UXO  is  being  cleared. 

The  question  of  labeling  false  alarms  is  of  more  than  academic  interest.  In  a 
commercial  sweep  of  an  area  near  the  Chocolate  Mountain  Bombing  Range,  393  anomalies 
were  detected,  4  of  which  proved  to  be  ordnance  items  (bombs).  NRL  personnel  were 
present  for  the  examination  of  116  anomalies,  97  of  which  had  their  source  identified  by 
excavation  and  19  of  which  did  not  Scaling  to  the  full  site,  there  were  4  ordnance  items, 
325  identified  nonordnance  anomalies,  and  64  mysterious  anomalies.  This  results  in  389 
detector  declarations  resulting  in  excavations  without  recovering  an  item  of  ordnance.  This 
is  the  number  of  false  alarms  according  to  the  definition  proposed.^ 

There  are  a  number  of  operational  issues  that  the  technology  demonstrations  could 
illuminate  to  stimulate  performance  of  further  analyses,  including: 

•  C^iantity  and  quality  of  support  equipment  and  infrastructure  required  to  main¬ 
tain  the  operability  of  the  sensors  in  the  field. 

•  The  number  of  support/maintenance  personnel  required  (both  DoD  and  con¬ 
tractor)  for  sustained  operation  of  the  sensors. 

•  Logistics  associated  with  the  movement  of  the  sensors  and  supporting  mfra- 
suiicture  from  one  CONUS  site  to  another  (deployability). 

•  Training  requirements/integration  of  sensor  into  existing  EOD  courses  for 
operators  and  maintainers. 

B .  DATA  COLLECTION  AND  MEASURES  OF  PERFORMANCE 

It  is  important  to  develop  protocols  for  data  collection  and  reporting  that  allow 
appropriate  comparisons  and  contrasts  among  different  demonstrations  to  be  made.  In 
some  instances,  different  demonstrations  will  seek  answers  to  questions  which  result  in 
data  that  cannot  be  readily  compared  to  data  from  other  demonstrations.  In  other  instances, 
however,  different  demonstrations  will  explore  the  same  phenomena,  resulting  in  data  that 


1  McDonald  J  R.,  and  R.  Robertson,  Evaluation  of  the  Trimble  DGPS  Navigation  System,  Chocolate 
Mountains  Bombing  Range:  A  Magnetometer  Survey  for  a  Gas  Transmission  Une  Southern 
California  Gas  Company;  Project  6902,  Naval  Research  Laboratory,  NRL/PU/61 10-94-260,  July  8, 
1994,  pp.  44  46. 
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lends  itself  to  cross-demonstration  comparison  and  analysis.  The  following  discussion 
seeks  to  structure  questions  and  answers  so  that  data  can  be  gathered  in  ways  that  are 
useful  to  both  users  and  developers  whose  perspectives  on  technology  demonstrations  may 
be  very  different 

There  are  two  principal  types  of  tests  of  a  detection  system:  (1)  tests  designed  to 
gather  <tata  on  the  operational  characteristics  of  the  sensor  to  produce  better  understanding 
of  its  performance,  and  (2)  tests  designed  to  gather  data  on  the  operational  peTformance  and 
techniques  of  sensor  employment  to  determine  its  utility.  The  information  requirements  for 
each  type  of  test  are  very  different. 

Tests  of  operational  characteristics  (sensor  phenomenology)  require  extensive 
information  about  the  locations  and  types  of  test  objects,  as  well  as  the  local  geology  and 
conditions,  so  that  the  sensor  response  can  be  understood. 

Tests  of  operational  performance  and  techniques  of  sensor  employment  require  that 
the  operator  have  no  information  about  the  ground  truth  and  that  there  be  no  interaction 
between  the  demonstrator  and  the  evaluator,  so  that  the  utility  of  the  sensor  can  be 
evaluated  without  bias.  We  will  concentrate  primarily  on  a  test  of  the  sensor  utility. 

There  are  three  reasons  for  emphasizing  tests  of  operational  performance  and 
technique: 

1.  The  ESTCP  goals  emphasize  demonstration  of  field-ready  technologies;  even 
if  enhanced  UXO  detection  technology  is  not  sufficiently  advanced  for 
inunediate  transition  to  the  field,  the  program  should  pull  the  technology 
towards  this  goal  by  virtue  of  test  design  and  evaluation. 

2.  UXO-qualified  personnel  need  performance  data  and  operational  information 
to  better  judge  the  utility  of  a  technology  or  system  for  their  jobs;  scientific  or 
engineering  data  on  sensor  performance  is  not  immediately  useful  or  relevant 
to  users. 

3.  Tests  focusing  on  sensor  phenomenology  often  require  use  of  proprietary  data 
or  other  data  closely  held  by  the  developers  and  their  government  sponsors, 
which  makes  widespread  analysis  and  discussion  of  the  test  results  more 
difficult 

The  most  important  measures  of  performance  of  UXO  detection  systems  are  the 
following: 

•  Probability  of  detection. 
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•  Ability  to  discriminate  ordnance  and  nonordnance  items  among  detected 
targets. 

•  False-alarm  rates. 

A  high  probability  of  detection  capability  is  important  in  the  efficacy  and  efficiency 
of  a  UXO  cleanup:  obviously,  locating  more  ordnance  allows  one  to  mitigate  more  risk. 
Discrimination  ability  and  false-alarm  rates  affect  the  amount  of  time  spent  at  one  site,  the 
number  of  excavations  performed,  and  the  overall  cost  of  a  cleanup.  To  clean  a  site, 
resources  must  be  expended  to  destroy  UXO  in  place  or  to  remove  it.  Any  resources 
expended  removing  debris  or  investigating  other  false  alarms  are,  in  a  sense,  wasted.  The 
EOD  personnel  interviewed  indicated  that  good  discrimination  ability  could  have  a  large 
impact  on  the  efficiency  of  most  cleanups,  because  well  over  half  of  the  objects  they 
investigate  are  not  intact  ordnance. 

There  are  a  number  of  variables  that  may  affect  the  detection  abUity  and  false-alarm 
rate  of  a  sensor.  Among  these  variables  are  the  following: 

•  Size,  shape,  and  condition  of  UXO. 

•  Attitude  of  UXO  and  depth  of  burial. 

•  Geographical  and  geological  conditions  surrounding  UXO. 

•  Past  and  current  human  activity  in  the  vicinity  of  UXO. 

•  Operating  conditions  at  the  time  UXO  reconnaissance  is  undertaken. 

The  effects  of  these  variables  on  sensor  operation  and  technique  need  to  be 
quantified.  For  example,  some  sensors  may  be  better  suited  to  detecting  certain  types  of 
ordnance  than  others.  Some  sensors  may  perform  better  on  UXO  that  is  oriented  closer  to 
the  horizontal  than  the  vertical,  regardless  of  the  type  of  UXO  in  question.  Some  sensors 
might  work  better  in  certain  soil  conditions  than  in  others;  some  sensors  might  work  better 
than  others  in  areas  where  there  is  a  great  deal  of  “cluttef’  comprised  of  man-made 
nonordnance  items.  Some  sensors  might  be  better  at  discriminating  among  detected  items, 
while  others  might  be  better  at  merely  detecting  the  presence  of  an  object  without 
necessarily  being  particularly  good  at  discriminating  among  targets. 

To  establish  the  reproducibility  of  results  across  many  conditions,  tests  need  to  be 
done  in  a  variety  of  conditions  representative  of  areas  where  the  sensor  is  proposed  for  use. 
Furthermore,  the  consistency  and  reliability  of  the  sensor,  in  general  and  under  various 
operating  conditions,  is  important  Data  about  failure  mode,  time  between  failure,  and  the 
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time  required  for  repair  should  also  be  coUected,  preferably  while  the  sensor  is  being  used 
by  UXO-qualified  personnel. 

The  capability  of  a  sensor  to  classify  targets  when  multiple  target  types  are  present 
is  next  in  importance  after  probability  of  detection  and  false-alarm  rate.  Although  classi¬ 
fication  is  likely  to  be  problematic  for  aU  of  the  systems  currently  being  developed  under 
ESTCP,  sensor  performance  data  should  be  collected  and  reported  in  a  standard  fashion. 
The  analog  to  Pd  and  false  alarm  rate  (P/a)  for  target  classification  is  the  so-called 
confusion  matrix.  As  illustrated  in  Figure  2,  the  columns  indicate  the  demonstrator  s 
identification  of  ordnance  type.  The  rows  indicate  true  ordnance  type.  In  the  example 
presented,  the  demonstrator  attempted  to  distinguish  bombs  and  mortar  rounds.  Five 
bombs  were  detected.  Of  these,  four  were  identified  as  bombs  and  one  was  identified  as  a 
mortar  round.  Ten  mortar  rounds  were  detected.  Seven  were  identified  as  mortars  and 
three  were  identified  as  bombs. 


Called  Bomb 

Called  Mortar  Round 

Is  a  Bomb 

4  (80%) 

1  (20%)* 

Is  a  Mortar  Round 

3  (30%) 

7  (70%)* 

*  Row  entries  must  sum  to  100%. 

Figure  2.  Illustrative  Confusion  Matrix  for  UXO  Target  Classification 


Location  and  navigation  ability  wUl  influence  the  resources  that  are  consumed  in 
investigating  each  identifled  UXO  item.  The  ability  to  accurately  locate  a  target,  both  in 
horizontal  space  and  in  distance  below  the  surface,  may  also  help  in  choosing  the  appro- 
priate  tools  for  excavating,  inspecting,  and  removing  the  detected  item.  Further,  the  ability 
of  a  sensor  to  resolve  closely  spaced  targets  is  important  UXO-qualified  personnel  need  to 
know  whether  the  anomaly  is  a  single  object  or  a  group  of  objects;  if  a  group  of  objects  are 
in  the  target,  knowledge  of  how  many  objects  are  in  the  target  is  also  important 

C.  DATA  ANALYSIS 

Determining  probability  of  detection  and  false-alarm  rate  requires  a  method  of 
scoring  demonstrator  declarations  (of  locations  believed  to  contain  ordnance)  to  separate 
detections  from  false  alarms.  This  can  be  a  nontrivial  issue  if  false  alarms  are  dense  or  if  a 
system,  even  with  good  detection  capability,  suffers  from  poor  navigation. 

One  can  envision  several  ways  of  pairing  up  targets  identified  by  the  demonstrator 
as  ordnance  with  emplaced  items.  The  entire  field  can  be  optimized  by  pattern  matching. 


IV-5 


where  the  detections  and  the  emplaced  items  are  overlaid  to  produce  the  maximum  overlap. 
This  is  straightforward  if  false  alarms  are  sparse,  but  difficult  if  they  are  frequent 
Furthermore,  such  pairing  ignores  a  demonstrator’s  absolute  ability  to  locate  targets. 
Locating  targets  precisely  is  very  challenging,  especially  for  airborne  platforms. 

The  other  option  is  to  choose  an  absolute  requirement  of  separation  distance.  Only 
those  declarations  that  fall  within  this  separation  distance  from  an  emplaced  item  would  be 
counted  as  a  detection.  This  has  a  more  stringent  location  requirement,  which  may  be 
difficult  for  airborne  platforms  to  meet.  Also,  it  will  generally  confuse  the  measure  of 
detection  ability  with  navigation  accuracy. 

To  understand  the  performance  of  the  sensor  over  its  range  of  operation,  a  receiver 
operating  characteristic  (ROC)  curve  is  often  employed.  The  ROC  curve  is  determined  by 
measuring  probability  of  detection  and  false-alarm  rate  at  several  thresholds  of  the  sensor’s 
sensitivity.  Good  sensor  performance  means  large  increases  in  probability  of  detection  for 
modest  increases  in  false  alarms  at  initial  sensor  performance  threshold.  As  the  sensor 
sensitivity  thresholds  are  increased,  there  is  a  crossover  in  performance.  Small  increases  in 
the  probability  of  detection  result  in  increasingly  larger  false-alarm  rates.  At  some  point, 
even  small  increases  in  sensor  sensitivity  result  in  only  minimal  increases  in  the  probability 
of  detection  but  very  large  increases  in  false  alarms.  With  a  ROC  curve  one  can  compare 
two  sensors  over  their  operational  range  without  worrying  about  whether  the  results  are 
dependent  on  different  threshold  settings  for  the  two  sensors.  Furthermore,  a  ROC  curve 
will  be  more  broadly  applicable  in  assessing  utility  since  different  regions  of  the  ROC  curve 
will  be  important  for  different  clearance  activities. 

D .  RECOMMENDED  MEASURES  OF  PERFORMANCE 

In  light  of  the  above  concerns,  we  recommend  use  of  the  following  measures  of 
performance:  probabdity  of  detection,  false-alarm  rate,  discriminant  capability,  location 
accuracy,  and  system  reliability/maintainability.  We  realize  that  individual  tests  may  not 
support  all  of  these  measures  and  that  additional  measures  may  also  be  appropriate  for 
certain  tests.  However,  this  core  of  measures  (with  terms  defined  below)  will  produce  data 
that— although  it  may  not  perfectly  reflect  the  performance  of  a  given  sensor  in  a  way  that 
is  agreeable  to  all — ^will  be  comparable  among  tests. 

•  Probability  of  Detection:  This  measure  is  needed  as  a  function  of  ordnance 
type  and  size,  as  a  function  of  depth,  and  as  a  function  of  geology/site  charac¬ 
teristics.  In  all  cases  the  data  should  be  recorded  as  a  function  of  a  calibrated 
sensor  threshold  or  confidence  level  for  highly  processed  returns.  The  scoring 
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needs  to  be  calculated  relative  to  known  ground  truth  (discussed  below). 
Declarations  of  nonordnance  items,  be  they  man-made  or  natural,  should  count 
as  false  alarms  (see  below).  Finally,  if  the  survey  speed  is  not  constrained, 
then  probability  of  detection  {Pd)  should  be  evaluated  as  a  function  of  survey 
speed.  For  visual  surface  sweeps,  increasing  the  survey  rate  has  been  shown 
to  result  in  reduced  probability  of  detection.^  It  would  be  useful  to  have 
similar  data  for  instrument  detection  of  buried  ordnance. 

•  False-Alarm  Rate:  The  primary  means  of  reporting  false  alarms  should  be  per 
unit  area,  FAAon.  This  data  should  be  taken  as  a  function  of  all.  the  same 
parameters  as  probability  of  detection.  This  can,  in  a  model-dependent  way, 
be  converted  to  a  probability  of  false  alarm  (P/a),  where  the  model  is  a  means 
of  specifying  how  many  opportunities  for  a  false  alarm  exist  per  unit  area.  All 
nonordnance  items  should  be  regarded  as  false  alarms,  even  if  they  are  man¬ 
made  objects  which  could  be  a  priori  known  to  mimic  the  signature  of  an 
ordnance  item.  The  purpose  of  the  exercise  is  to  assess  the  efficiency  of  die 
search,  not  to  check  on  the  laws  of  physics.  However,  in  order  to  extrapolate 
results  to  other  sites,  distinguishing  man-made  objects  and  naturally  occurring 
objects  as  sources  of  false  alarms  is  likely  to  be  useful.  In  addition,  simple 
calibrated  surveys  of  areas  believed  to  be  uncontaminated  with  ordnance  will 
be  valuable  in  estimating  performance  at  other,  uncontrolled  sites. 

•  Discrimination  Capability:  The  most  useful  discrimination  capability  would  be 
the  ability  to  discriminate  hazardous  from  nonhazardous  items.  With  the 
possible  exception  of  thermal  neutron  activation,  we  are  aware  of  no  technical 
approaches  within  the  ESTCP  projects  that  have  any  prospect  in  this  area.  The 
next  most  useful  discrimination  ability  would  be  between  ordnance  items, 
ordnance-related  debris,  and  nonordnance  items.  Finally,  within  the  category 
of  ordnance,  it  would  be  desirable  to  discriminate  one  ordnance  type  from 
another.  This  type  of  discrimination  is  usually  referred  to  as  classification.  It 
is  our  belief  that  the  only  discrimination  ability  likely  to  be  demonstrated  will 
be  related  to  physical  size. 

•  Location  Accuracy:  For  location  accuracy,  horizontal  position  and  depth  are 
important  In  addition,  it  is  important  to  be  able  to  place  detections  on  an 
absolute  coordinate  system  or  in  relation  to  a  fixed  landmark  as  well  as  relative 
to  the  sensor  platform.  With  the  proliferation  of  digital  Global  Positioning 
System  (DGPS)  technologies,  this  is  becoming  increasingly  straightforward. 

•  System  Reliability/Maintainability:  Records  should  be  maintained  describing 
the  manner  in  which  demonstrated  systems  fail,  the  amount  of  time  between 


2  NAVEODFAC  Technical  Report  TR-23S,  “Study  of  Search  Effectiveness  of  Surface  Oeaiance 
Techniques  on  Kahoolawe  Island”  (Indian  Head,  Maryland:  November  1980). 
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failures,  and  the  dollar  cost  needed  to  return  the  system  to  working  order. 
Cost  and  mean-time-to  repair  data  on  prototype  systems  should  be  viewed  with 
great  skepticism;  failure  mode  data  is  especially  important  to  collect  The 
reliability  and  maintainability  of  commercially  available  components  integrated 
into  the  technology  demonstration  is  also  important  to  track.  The  contractors 
should  be  required  to  prepare  detailed  failure  mode  analysis  of  prototype 
system  components  and  subsystems,  relying  upon  the  databases  of  reliability 
maintained  by  the  Reliability  Analysis  Center,  Rome,  New  York,  to 
illuminate  potential  system  design  issues  for  further  consideration. 

E.  POSTMORTEM  ANALYSES 

A  significant  unknown  likely  to  remain  so  at  the  end  of  the  demonstration  is  the 
source  of  false  alarms.  After  the  demonstration  is  complete,  it  will  be  useful  to  examine  as 
many  false  alarms  as  possible  to  determine  their  common  features,  to  identify  the  source(s) 
of  those  with  the  largest  signatures,  and  to  identify  the  source(s)  of  the  most  frequent  false 
alarms.  A  detailed  calibrated  map  of  sensor  returns  and  a  calibrated  map  of  ground  truth 
items  that  allow  for  matching  are  very  much  needed  in  order  to  better  understand  the  origin 
of  false  alarms  across  the  entire  range  of  demonstrated  systems  and  technologies. 

False-alarm  data,  if  archived,  will  allow  for  two  additional  valuable  activities. 
First,  other  processing  schemes  can  be  applied  to  the  data  to  see  if  performance  can  be 
improved.  The  value  of  the  demonstration  may  be  greatly  increased  if  the  final  perform¬ 
ance  measures  are  not  constrained  by  processing  schemes  in  place  before  the  first  data  are 
taken.  Second,  if  similar  background  data  are  taken  on  a  variety  of  uncontaminated  sites, 
performance  can  be  more  widely  extrapolated.  These  two  activities  are  linked.  To  have 
credibility,  processing  schemes  developed  after  ground  truth  data  from  a  specific  site  have 
been  revealed  to  the  developer  must  not  be  “site-specific,”  but  rather  shown  to  be  applicable 
to  sites  not  used  in  algorithm  adjustment. 
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V.  SUMMARY  AND  CONCLUSIONS 


Tests  and  demonstrations  of  developmental  equipment  largely  serve  the  purpose  of 
contirming  that  the  equipment  performs  as  the  developers  intended  in  the  scenarios  they 
had  envisioned.  This  is  a  necessary  portion  of  any  successful  developmental  program. 

The  ESTCP  UXO  Detection  Demonstration  Project  seeks  to  lay  the  groundwork  for 
future  acquisition  of  UXO  detection  equipment  that  goes  beyond  use  of  magnetometer 
technology.  As  noted  by  the  Defense  Science  Board,  the  General  Accounting  Office,  the 
DoD  Inspector  General,  and  even  the  DoD,  UXO  detection  within  CONUS,  focused  on 
relatively  large,  metallic  ordnance  remains  is  a  very  difficult  task.  The  expansion  of  this 
effort  to  include  global  search  and  detection  capability,  especially  to  support  detection, 
discrimination,  and  removal  of  plastic  landmines  is  far  beyond  the  limited  scope  of  the 
proposed  demonstrations.  Within  the  proposed,  limited  scope,  two  additional  RDT&E 
purposes  can  be  served  with  relatively  little  increased  effort  at  this  stage.  These  are 
discussed  below. 

A.  EXPANDED  UXO  DEMONSTRATION  PURPOSES 

The  first  of  these  purposes  is  aimed  primarily  at  the  technical  community.  Data 
collection  on  UXO  detection  sensor  technical  performance  should  allow  for  common 
definitions,  figures  of  merit,  and  analytic  approaches  across  all  the  demonstrations  to 
facilitate  technology  comparisons.  In  addition,  data  collection  on  backgrounds  in  land 
uncontaminated  with  UXO  would  add  gready  to  the  community's  ability  to  extrapolate 
performance  to  other  sites. 

The  second  expanded  RDT&E  purpose  is  aimed  primarily  at  the  user  and 
acquisition  communities.  The  sensor  technical  performance  data  collected  and  the  system 
field  performance  should  document  detections,  false  alarms,  mean  time  between  failures, 
mean  time  to  repair,  and  other  data  on  figures  of  merit  of  interest  to  the  user  community.  In 
the  case  of  UXO  detector  technology,  this  user  community  is  primarily  the  DoD  EOD 
community  and  its  associated  civilian  contractors,  who  provide  UXO-qualified  personnel  to 
DoD  ranges,  and  other  DoD  contractors  engaged  in  environmental  cleanup  activities.  The 
collection  and  documentation  of  system  performance  based  on  operator  techniques  as  well 
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as  sensor  performance  would  inform  program  managers  seeking  to  make  decisions  on 
which  technical  approaches  to  pursue  or  field. 

B .  ISSUES  FOR  THE  TECHNICAL  COMMUNITY 

Sensor  systems  are  generally  characterized  in  terms  of  a  ROC,  which  relates 
increases  in  false-alarm  rates  to  increases  in  the  probability  of  detection  as  the  catena  for 
declaring  a  detection  are  relaxed.  Technical  tests  measuring  probability  of  detection  and 
false-alarm  rates  inevitably  involve  the  use  of  surrogate  targets.  Most  UXO  detection 
schemes,  including  most  ESTCP  projects,  rely  on  the  detection  of  metals  or  electrical 
discontinuities  that  might  be  associated  with  ordnance,  rather  than  trying  to  detect  directly 
the  presence  of  explosive.  Consequently,  there  is  room  for  debate  about  how  to  count 
different  objects  as  false  alarms  or  true  detections  in  a  demonstration  that  uses  intact  inert 
ordnance  in  place  of  live  ordnance  in  various  stages  of  damage  or  decay.  The  distinction  of 
interest  to  the  EOD  technicians  is,  “Is  this  an  object  which  presents  a  possible  explosive 
hazard,  and  therefore  requires  EOD  examination,  or  can  the  object  be  bypassed  for  later 
examination  by  non-UXO-qualified  persons?”  This  distinction  does  not  truly  apply  when 
using  explosive  simulants  for  safety  reasons. 

We  believe  the  most  useful  approach  is  to  declare  as  false  alarms  anything  other 
than  ordnance  items.  From  an  operational  point  of  view,  any  resources  spent  on  non- 
hazardous  items  are  wasted.  If  possible,  it  is  useful  to  distinguish  man-made  nonordnance 
items  from  “natural”  false  alarms  because  this  will  relate  to  extrapolating  results  to  other 
locations.  The  extrapolation  will  be  greatly  aided  by  background  data  taken  on 
uncontaminated  sites. 

After  probability  of  detection  and  false-alarm  rate,  the  classification  capabUity  of  a 
sensor  when  there  are  multiple  target  types  is  of  interest.  We  are  skeptical  of  the  classi¬ 
fication  capability  of  any  of  the  systems  currendy  being  developed  under  ESTCP;  never¬ 
theless,  we  believe  the  data  should  be  reported  in  the  form  of  a  confusion  matrix  (see 
Chapter  IV). 

Finally,  in  addition  to  the  above,  background  clutter  maps  for  regions  believed  to  be 
free  of  explosive  contamination  should  be  collected,  preferably  for  several  varied  areas. 
This  information  is  necessary  to  determine  whether  the  sensor  performance  at  the  test  site  is 
characteristic  of  its  performance  anywhere  or  is  idiosyncratic. 
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To  satisfy  the  main  needs  for  the  technical  community: 

•  Specify  detection  capability  by  full  ROC  curves,  with  the  calibrated  thresholds 
used  for  the  probability  of  detection  and  probability  of  false  alarm  points 
recorded.  All  nonordnance  items  declared  should  be  counted  as  false  alarms. 

•  Characterize  effect  on  ROC  curve  of  ordnance  type — include  survey  rate, 
location,  etc. 

•  Characterize  classification  capability  by  the  full  confusion  matrix,  also  as  a 
function  of  threshold. 

•  Collect  calibrated  background  data  on  uncontaminated  sites  to  the  maximum 
extent  feasible. 

C.  ISSUES  FOR  THE  USER  COMMUNITY 

For  the  users  of  UXO  detectors,  the  issue  of  utility  extends  beyond  the  sensor 
systems’  technical  performance.  The  ease  of  operation,  reliability  of  the  equipment,  and 
training  required  to  achieve  and  maintain  proficiency  are  of  interest.  Furthermore,  it  must 
be  possible  to  operate  the  equipment  in  a  manner  consistent  with  established  procedures 
and  sound  safety  practices.  The  goal  of  ESTCP  is  to  meet  the  broadest  DoD  environmental 
monitoring  and  restoration  technology  requirements.  Therefore,  the  demonstration  should 
be  documented  so  that  those  approaches  which  are  most  promising  can  be  examined  further 
to  determine  whether  or  not  the  technology  can  be  repackaged  for  use  in  combat  EOD 
environments  as  well  as  in  seeking  UXO  in  the  peacetime,  CONUS  environment.  The 
issues  of  suitability — training,  consistency  with  current  operations,  and  deployability — 
could  be  addressed  by  having  EOD  personnel  participate  in  one  or  more  of  the  demon¬ 
strations  with  the  equipment.  This  might  require  an  additional  training  effort,  but  the 
feedback  from  user  to  developer  would  be  invaluable.  Estimating  costs  of  production 
versions  of  developmental  systems  is  an  uncertain  practice.  The  needed  information  on 
system  complexity,  maturity  of  components,  scheduled  maintenance,  mean  time  between 
failure,  and  so  on  could  be  made  available  by  the  developer  for  use  in  standard  estimation 
techniques.  To  satisfy  the  main  information  and  technology  evaluation  needs  for  the  user 
community,  the  demonstration  should 

•  Include  active  duty  EOD  technicians  in  the  demonstrations,  either  operating 
equipment  or  interpreting  sensor  output  or  both.  This  will  provide  insight  into 
the  suitability  of  the  equipment,  especially  as  it  affects 

—  Ease  of  operation, 

—  Training  requirements,  and 
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—  Consistency  with  current  safety  practices. 

•  Provide  system  data  that  will  allow  independent  cost  estimation  for  procure¬ 
ment  and  operating  costs. 

•  Result  in  the  collection  and  analysis  of  data  that  sheds  light  on  the  interaction 
between  UXO  detectors  and  munitions  fuzes. 

D.  RECOMMENDATION 

The  collection  of  consistent  data  on  effectiveness,  suitability,  and  cost,  as  outlined 
above,  should  be  pursued  as  part  of  the  ESTCP  program.  This  data  will  be  essential  for 
evaluation  and  comparison  of  different  technologies.  It  will  also  prove  essential  to  decision 
makers  in  determining  which  technologies  or  systems  to  pursue  further.  In  addition,  the 
interaction  with  the  user  community  in  this  fashion  will  greatly  increase  the  probability  that 
an  effective  system  is  ultimately  placed  in  the  user’s  hands. 
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